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Preface 


Attributions 


The main inspiration for this book, as well as the vast majority of the source material, is 
Notes on Diffy Qs by Jitf Lebl [JL]. The fact that the book is freely available and open-source 
provided the main motivation for creating this current text. It allowed this book to be put 
together in a timely manner to be useful. It significantly reduced the work needed to put 
together a free textbook that fit the course exactly. 


Introduction to this Version 


This text was originally designed for the Math 244 class at Rutgers University. This class is a 
first course in Differential Equations for Engineering majors. This class is taken immediately 
after Multivariable Calculus and does not assume any knowledge of linear algebra. Prior 
to the design of this book, the course used Boyce and DiPrima’s Elementary Differential 
Equations and Boundary Value Problems |BD]. The course provided a very brief introduction 
to matrices in order to get to the information necessary to handle first order systems of 
differential equations. With the course being redesigned to include more linear algebra, I 
was pointed in the direction of Jitf Lebl’s Notes on Diffy Qs [JL], which was meant to be a 
drop-in replacement for the Boyce and DiPrima text, and as of a more recent version of the 
text, contained an appendix on Linear Algebra. 

In creating this book, I wanted to retain the style of Notes on Diffy Qs [JL] but shape 
the text into something that directly fit the course that we wanted to run. This included 
reorganizing some of the topics, extra contextualization of the concept of differential equations, 
sections devoted to modeling principles and how these equations can be derived, and guidance 
in using MATLAB to solve differential equations numerically. Specifically, the content added 
to this book is 


e Appendix A that gives an introduction or review to coding in MATLAB, as well as 
references to sample MATLAB files that can be used to easily sketch slope fields and 
solution curves to differential equations. 


e Section 1.10 on the accumulation equation and its use in mathematical models, and 
§ 1.11 which contains a discussion of parameter estimation, with inspiration taken from 
SIMIODE. 
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e The work on the eigenvalue method was split into three sections to account for real, 
complex, and repeated eigenvalues. 


e A discussion of the trace-determinant plane and applications to analysis of linear (and 
non-linear) systems was added in § 4.7. 


e Appendix B on prerequisite material to be referred to when needed. Some of the 
material here was pulled from Stitz and Zeager’s book Precalculus [SZ]. 


e Exercises were added at the end of most sections of the text. 
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I would like to acknowledge David Molnar, who initially referred me to the Notes on Diffy 
Qs text [JL], as well as the Precalculus text [SZ], and provided inspiration and motivation 
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Introduction to Notes on Diffy Qs 


This book [JL] originated from my class notes for Math 286 at the University of Illinois at 
Urbana-Champaign (UIUC) in Fall 2008 and Spring 2009. It is a first course on differential 
equations for engineers. Using this book, I also taught Math 285 at UIUC, Math 20D at 
University of California, San Diego (UCSD), and Math 4233 at Oklahoma State University 
(OSU). Normally these courses are taught with Edwards and Penney, Differential Equations 
and Boundary Value Problems: Computing and Modeling [EP], or Boyce and DiPrima’s 
Elementary Differential Equations and Boundary Value Problems [BD], and this book aims 
to be more or less a drop-in replacement. Other books I used as sources of information 
and inspiration are E.L. Ince’s classic (and inexpensive) Ordinary Differential Equations |I], 
Stanley Farlow’s Differential Equations and Their Applications |F], now available from Dover, 
Berg and McGregor’s Elementary Partial Differential Equations |BM], and William Trench’s 
free book Elementary Differential Equations with Boundary Value Problems |T]. See the 
Further Reading chapter at the end of the book. 


Computer resources 
The book’s website https: //www. jirka.org/diffyqs/ contains the following resources: 
1. Interactive SAGE demos. 


2. Online WeBWork homeworks (using either your own WeBWorkK installation or Edfinity) 
for most sections, customized for this book. 


3. The PDFs of the figures used in this book. 


I taught the UIUC courses using IODE (https://faculty.math.illinois.edu/iode/). 
IODE is a free software package that works with Matlab (proprietary) or Octave (free 
software). The graphs in the book were made with the Genius software (see https: //www. 
jirka.org/genius.html). I use Genius in class to show these (and other) graphs. 
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Chapter 0 


Introduction 


0.1 Introduction to differential equations 


Attribution: [JL], §0.2. 


After this section, you will be able to: 


e Identify a differential equation and determine the order of a differential equation, 
e Verify that a function is a solution to a differential equation, and 
e Solve some fundamental differential equations. 


0.1.1 Differential equations 
Consider the following situation: 


An object falling through the air has its velocity affected by two factors: gravity 
and a drag force. The velocity downward is increased at a rate of 9.8 m/s? due 
to gravity, and it is decreased by a rate equation to 0.3 times the current velocity 
of the object. If the ball is initially thrown downwards at a speed of 2 m/s, what 
will the velocity be 10 seconds later? 


There might be enough information here to determine the velocity at any later point in 
time (it turns out, there is) but the information given isn’t really about the velocity. Rather, 
information is given about the rate of change of the velocity. We know that the velocity 
will be increased at a rate of 9.8m/s? due to gravity. How can this be interpreted? The 
rate of change has been discussed previously way back in Calculus 1; this is the derivative. 
Thus, if we let the unknown function v(t) represent the velocity of the object, the description 
above gives information about the derivative of this function foru(t). Taking the two different 
factors (the increase and decrease of velocity) into account, we can write an expression for 
this derivative, giving that 
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Even though we don’t know what v(t) is, we know that it must affect the derivative of the 
velocity in this particular way, so we can write this equation. That’s why we give a name to 
this function, so that we can use it in writing this equestion, which, since it is an equation 
involving the derivative of an unknown function v(t), we call this a differential equation. Our 
goal here would be to use this information, plus the fact that the velocity at time zero is 
v(0) = 2 m/s to find the value of v(10), or, more generally, the function v(t) for any t. 

The laws of physics, beyond just that of simple velocity, are generally written down as 
differential equations. Therefore, all of science and engineering use differential equations 
to some degree. Understanding differential equations is essential to understanding almost 
anything you will study in your science and engineering classes. You can think of mathematics 
as the language of science, and differential equations are one of the most important parts 
of this language as far as science and engineering are concerned. As an analogy, suppose 
all your classes from now on were given half in Swahili and half in English. It would be 
important to first learn Swahili, or you would have a very tough time getting a good grade in 
your classes. Without it, you might be able to make sense of some of what is going on, but 
would definitely be missing an important part of the picture. 


Definition 0.1.1 


A differential equation is an equation that involves one or more derivatives of an 


unknown function. For a differential equation, the order of the differential equation is 
the highest order derivative that appears in the equation. 


One example of a first order differential equation is 


ba = 2eost. (1) 
Here x is the dependent variable and t is the independent variable. Note that we can use 
any letter we want for the dependent and independent variables. This equation arises from 
Newton’s law of cooling where the ambient temperature oscillates with time. 

To make sure that everything is well-defined, we will assume that we can always write 
our differential equation with the highest order derivative written as a function of all lower 
derivatives and the independent variable. For the previous example, since we can write (1) as 


d 
ea = 2cost—2 


dt 


where the highest derivative x’ is written as a function of t and x, we have a proper differential 
equation. On the other hand, something like 


(#) +¥- 1 (2) 


is not a proper differential equation because we can’t solve for ay This expression could 
either be written as F d 
y y 
ae | ee BM gf] age 
dt z se dt oes 
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and while both of these are proper differential equations, the version in (2) is not. 

For some equations, like y’ = y?, the independent variable is not explicitly stated. We 
could be looking for a function y(t) or a function y(zx) (or y of any other variable) and without 
any other information, any of these is correct. Usually, there will be information in the 
problem statement to indicate that the independent variable is something like time, in which 
case everything should be written in terms of t. It is for this reason that Leibniz notation is 
preferred for derivatives; an equation like 


dy _ 
dt 
is unambiguously looking for any answer y(t). 


Example 0.1.1: All of the below are differential equations 


—=ecy z+ 2% =tsinz 


re aa —7 yl" ua (y")? _ 3y — t. 


Note that any letter can be used for the unknown function and its dependent variable. From 
the context of the equations, we can see that the unknown functions we are looking for in 
these examples are y(t), z(t), y(x), and y(t) respectively. The order of these equations are 1, 
2, 4, and 3 respectively. 


0.1.2 Solutions of differential equations 


Solving the differential equation means finding the function that, when we plug it into the 
differential equation, gives a true statement. For example, take (1) from the previous section. 
In this case, this means that we want to find a function of t, which we call x, such that when 
we plug x, t, and de into (1), the equation holds; that is, the left hand side equals the right 
hand side. It is the same idea as it would be for a normal (algebraic) equation of just x and 
t. We claim that 
x = x(t) =cost+sint 
is a solution. How do we check? We simply plug x into equation (1)! First we need to 
compute “2. We find that “ = — sint + cost. Now let us compute the left-hand side of (1). 
dx 


a +a =(-—sint+ cost) +(cost+sint) = 2cost. 
—_— we OCP are~” 


dx x 
dt 
—t 


Yay! We got precisely the right-hand side. But there is more! We claim x = cost +sint+e 
is also a solution. Let us try, 


dx : 5 
— =-—sint+cost—e-. 


dt 
We plug into the left-hand side of (1) 


d. 
7 +a = (-sint + cost —e*) + (cost + sint +e") = 2cost. 
—-_-_-eo rr wr” eee. 


dx x 
dt 
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And it works yet again! 
So there can be many different solutions. For — . 
this equation all solutions can be written in the form 


x =cost+sint+Ce*, 


for some constant C’. Different constants C' will 
give different solutions, so there are really infinitely | °F 
many possible solutions. See Figure 1 for the graph 
of a few of these solutions. We do not yet know how 
to find this solution, but we will get to that in the 0 1 2 3 ‘ 5 
next chapter. 


Figure 1: Few solutions of oe +2 = 

Solving differential equations can be quite hard. | 2cost. 

There is no general method that solves every differ- 

ential equation. We will generally focus on how to get exact formulas for solutions of certain 
differential equations, but we will also spend a little bit of time on getting approximate 
solutions. And we will spend some time on understanding the equations without solving 
them. 

Most of this book is dedicated to ordinary differential equations or ODEs, that is, equations 
with only one independent variable, where derivatives are only with respect to this one variable. 
If there are several independent variables, we get partial differential equations or PDEs. 

Even for ODEs, which are very well understood, it is not a simple question of turning 
a crank to get answers. When you can find exact solutions, they are usually preferable to 
approximate solutions. It is important to understand how such solutions are found. Although 
in real applications you will leave much of the actual calculations to computers, you need to 
understand what they are doing. It is often necessary to simplify or transform your equations 
into something that a computer can understand and solve. You may even need to make 
certain assumptions and changes in your model to achieve this. 

To be a successful engineer or scientist, you will be required to solve problems in your job 
that you have never seen before. It is important to learn problem solving techniques, so that 
you may apply those techniques to new problems. A common mistake is to expect to learn 
some prescription for solving all the problems you will encounter in your later career. This 
course is no exception. 
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0.1.3 Differential equations in practice 


So how do we use differential equations in sci- Real-world problem 

ence and engineering? The main way this takes 

place is through the process of mathematical atrc/ \ mere 
modeling. First, we have some real-world prob- 

lem we wish to understand. We make some Niatheenaienl solve Metheiareal 
simplifying assumptions and create a mathe- scarlel BSTaiOn 
matical model, which is a translation of this 

real-world problem into a set of differential equations. Think back to the example at the 
beginning of this section. We took a physical situation (a falling object) with some knowledge 
about how it behaves and turned that into a differential equation that describes the velocity 
over time. Then we apply mathematics to get some sort of a mathematical solution. Finally, 
we need to interpret our results, determining what this mathematical solution says about the 
real-world problem we started with. For instance, in the example at the start of the section, 
we could find the function v(t), but then need to interpret that if we were to plug 10 into 
this function, we will get the velocity 10 seconds later. 

Learning how to formulate the mathematical model and how to interpret the results is 
what your physics and engineering classes do. In this course, we will focus mostly on the 
mathematical analysis. This will be interspersed with discussions of this modeling process to 
give some context to what we are doing, and give practice for what will be seen in future 
physics and engineering classes. 

Let us look at an example of this process. One of the most basic differential equations is 
the standard exponential growth model. Let P denote the population of some bacteria on a 
Petri dish. We assume that there is enough food and enough space. Then the rate of growth 
of bacteria is proportional to the population—a large population grows quicker. Let t denote 
time (say in seconds) and P the population. Our model is 


dP 
— =kP, 
dt 

for some positive constant k > 0. 


Example 0.1.2: Suppose there are 100 bacteria at time 0 and 200 bacteria 10 seconds later. 
How many bacteria will there be 1 minute from time 0 (in 60 seconds)? 


Solution: First we need to solve the equation. We claim that a solution is given by 
PO)=Ce™ 
where C' is a constant. Let us try: 


sad =Ofe" SEP 
dt 
And it really is a solution. 
OK, now what? We do not know C, and we do not know k. But we know something. We 
know P(0) = 100, and we know 
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P(10) = 200. Let us plug these conditions in and see what happens. 


SPO) =Ce"=C 
200 =.P(10) = 100e""". 


Therefore, 2 = ec!" or na —k~0.069. So 
P(t) = 100 e(ln2)t/10 ~ 1QQ ¢0-069t. 


At one minute, t = 60, the population 
is P(60) = 6400. See Figure 2. 

Let us talk about the interpretation of 
the results. Does our solution mean that 
there must be exactly 6400 bacteria on the 
plate at 60s? No! We made assumptions 
that might not be true exactly, just approx- 
imately. If our assumptions are reasonable, 
then there will be approximately 6400 bacte- 
ria. Also, in real life P is a discrete quantity, 
not a real number. However, our model has 
no problem saying that for example at 61 
seconds, P(61) = 6859.35. | 


Figure 2: Bacteria growth in the first 60 seconds. 


Normally, the k in P’ = kP is known, and we want to solve the equation for different 
initial conditions. What does that mean? Take k = 1 for simplicity. Suppose we want to 
solve the equation 4 = P subject to P(0) = 1000 (the initial condition). Then the solution 
turns out to be (exercise) 


P(t) = 1000e. 


We call P(t) = Ce’ the general solution, as every solution of the equation can be written 
in this form for some constant C’. We need an initial condition to find out what C’ is, in 
order to find the particular solution we are looking for. Generally, when we say “particular 
solution,” we just mean some solution. 


0.1.4 Four fundamental equations 


A few equations appear often and it is useful to know what their solutions are. Let us 
call them the four fundamental equations. Their solutions are reasonably easy to guess by 
recalling properties of exponentials, sines, and cosines. They are also simple to check, which 
is something that you should always do. No need to wonder if you remembered the solution 
correctly. It is good to have these as solutions that you “know” to build from when we learn 
solutions to other differential equations later on. In Chapter 1 we will cover the first two, 
and the last two will be discussed in Chapter 2. 


First such equation is 
dy 
—" = ky, 
da id 
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for some constant k > 0. Here y is the dependent and x the independent variable. The 
general solution for this equation is 


y(x) = Ce**. 


We saw above that this function is a solution, although we used different variable names. 
Next, 
dy 


dx = kay, 


for some constant k > 0. The general solution for this equation is 
yay=aCe ™. 
Exercise 0.1.1: Check that the y given is really a solution to the equation. 
Next, take the second order differential equation 
ay =—k*y 
dx” 
for some constant k > 0. The general solution for this equation is 


y(x) = Ci cos(ka) + Co sin(ka). 


Since the equation is a second order differential equation, we have two constants in our 
general solution. 


Exercise 0.1.2: Check that the y given is really a solution to the equation. 
Finally, consider the second order differential equation 
dg. 
da” 


for some constant k > 0. The general solution for this equation is 


ky, 


ye) = Cie” # Cye—™, 
or 
y(x) = D, cosh(kx) + D2 sinh(kz). 
For those that do not know, cosh and sinh are defined by 


erie e =e * 
cosh x = a sinh x = ———— 
2 2 
They are called the hyperbolic cosine and hyperbolic sine. These functions are sometimes 
easier to work with than exponentials. They have some nice familiar properties such as 


cosh 0 = 1, sinh 0 = 0, and ao cosh x = sinh (no that is not a typo) and ao sinh x = cosh a. 


Exercise 0.1.3: Check that both forms of the y given are really solutions to the equation. 
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Example 0.1.3: In equations of higher order, you get more constants you must solve for 
to get a particular solution. The equation “4 = 0 has the general solution y = Cyx + C9; 


simply integrate twice and don’t forget about the constant of integration. Consider the 
initial conditions y(0) = 2 and y'(0) = 3. We plug in our general solution and solve for the 
constants: 

2=y(0) =C),-04+C2.=Cr, Sy (0) Cas 


In other words, y = 3x + 2 is the particular solution we seek. 


0.1.5 Exercises 

Note: Exercises marked with * have answers in the back of the book. 

Exercise 0.1.4: Show that x = e* is a solution to xv” — 122" + 482' — 64x = 0. 
Exercise 0.1.5:* Show that x = e~*! is a solution to x" + 42’ + 42 = 0. 

Exercise 0.1.6: Show that x = e' is not a solution to x” — 12x" + 482’ — 64x = 0. 
Exercise 0.1.7: Is y = sint a solution to (az)” =l-y’? Justify. 


Exercise 0.1.8:* Is y = x? a solution to xy" — 2y = 0? Justity. 


Exercise 0.1.9: Let yy + 2y’ — 8y = 0. Now try a solution of the form y = e™ for some 
(unknown) constant r. Is this a solution for some r? If so, find all such r. 


Exercise 0.1.10:* Let xy" — y' =0. Try a solution of the form y = x". Is this a solution 
for some r? If so, find all such r. 


Exercise 0.1.11: Verify that x = Ce~” is a solution to x’ = —2x. Find C to solve for the 
initial condition x(0) = 100. 


Exercise 0.1.12: Verify that x = Cje~' + Cye” is a solution to x” — x’ — 2x = 0. Find C, 
and C, to solve for the initial conditions x(0) = 10 and x'(0) = 0. 


Exercise 0.1.13:* Verify that x = Cie’ + C2 is a solution to x" — x! = 0. Find C, and C, 
so that x satisfies x(0) = 10 and x'(0) = 100. 


Exercise 0.1.14: Find a solution to (2')* + x? = 4 using your knowledge of derivatives of 
functions that you know from basic calculus. 


Exercise 0.1.15:* Solve & = 8p and y(0) = —9. 


Ss 


Exercise 0.1.16: Solve: 


dA dH 
—— = — = 1 
a) 2 =-10A, AO) =5 b) = 3H, H(0) 
2 ax 
c) dae y(0)=0, y(0)=1 d) age —9r, x(0)=1, 2'(0)=0 


Exercise 0.1.17:* Solve: 
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dx d?x i 

dp arr 

= = — =47T, T(0)= T' (0) = 
c) i 3p, p(0)=4 d) 7a , T(0)=0, T’(0)=6 


Exercise 0.1.18: Is there a solution to y’ = y, such that y(0) = y(1)? 


Exercise 0.1.19: The population of city X was 100 thousand 20 years ago, and the popula- 
tion of city X was 120 thousand 10 years ago. Assuming constant growth, you can use the 


exponential population model (like for the bacteria). What do you estimate the population is 
now? 


Exercise 0.1.20: Suppose that a football coach gets a salary of one million dollars now, 
and a raise of 10% every year (so exponential model, like population of bacteria). Let s be 
the salary in millions of dollars, and t is time in years. 


a) What is s(0) and s(1). b) Approximately how many years will it 
take for the salary to be 10 million. 


c) Approximately how many years will it d) Approximately how many years will it 
take for the salary to be 20 million. take for the salary to be 30 million. 
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0.2 Classification of differential equations 


Attribution: [JL], §0.8. 


After this section, you will be able to: 


e Classify equation as ordinary or partial differential equations, 


e Identify whether an equation is linear or non-linear, and 
e Classify linear equations as homogenoeus, non-homogenoeus, or constant coeffi- 
cient, as appropriate. 


There are many types of differential equations, and we classify them into different categories 
based on their properties. Let us quickly go over the most basic classification. We already 
saw the distinction between ordinary and partial differential equations: 


Definition 0.2.1 


e Ordinary differential equations or (ODE) are equations where the derivatives are 
taken with respect to only one variable. That is, there is only one independent 
variable. 


e Partial differential equations or (PDE) are equations that depend on partial 
derivatives of several variables. That is, there are several independent variables. 


Let us see some examples of ordinary differential equations: 


d 

a = ky, (Exponential growth) 

d 

a =k(A-y), (Newton’s law of cooling) 
2 

ms ss ( +kx = f(t). (Mechanical vibrations) 


0 a) 

sr + cs = 0), (Transport equation) 

0 oP 

oT = — (Heat equation) 

gQ2 g2 g2 

oP = 53 a a (Wave equation in 2 dimensions) 


If there are several equations working together, we have a so-called system of differential 
equations. For example, 
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is asimple system of ordinary differential equations. Maxwell’s equations for electromagnetics, 


V:-D= P; V-B= 0, 
ab 
Ot’ Ot’ 

are a system of partial differential equations. The divergence operator V- and the curl 


operator Vx can be written out in partial derivatives of the functions involved in the z, y, 
and z variables. 


ae 


In the first chapter, we will start attacking first order ordinary differential equations, that 
is, equations of the form a = f(x,y). In general, lower order equations are easier to work 
with and have simpler behavior, which is why we start with them. 


We also distinguish how the dependent variables appear in the equation (or system). 
Definition 0.2.2 


We say an equation is linear if the dependent variable (or variables) and their derivatives 


appear linearly, that is only as first powers, they are not multiplied together, and no 
other functions of the dependent variables appear. Otherwise, the equation is called 
nonlinear. 


Another way to determine if a differential equation is linear is if the equation is a sum of 
terms, where each term is some function of the independent variables or some function of 
the independent variables multiplied by a dependent variable or its derivative. That is, an 
ordinary differential equation is linear if it can be put into the form 


d” a d 
Ont) > + Oy 1(@) +++ + ay (2) + ao(x)y = 0(2. (3) 
The functions ao, a1, ..., Gy are called the coefficients. The equation is allowed to depend 
arbitrarily on the independent variable. So 
ee. wat 9 - J 
nT t sintg)g +e =, (4) 
is still a linear equation as y and its derivatives only appear linearly. The equation 
d2 et 
= =f 
cos(z) i ts 


is also linear, even though it is not initially in the correct form. From this equation, we can 
move the last term over to the right-hand side as a —<, and then it is in the correct form, 
with the oe term missing (or has coefficient zero). 

All the equations and systems above as examples are linear. It may not be immediately 
obvious for Maxwell’s equations unless you write out the divergence and curl in terms of 
partial derivatives. Let us see some nonlinear equations. For example Burger’s equation, 
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is a nonlinear second order partial differential equation. It is nonlinear because y and ou are 
multiplied together. The equation 


dz 


is a nonlinear first order differential equation as there is a second power of the dependent 
variable x. 


Definition 0.2.3 


A linear equation may further be called homogeneous if all terms depend on the 


dependent variable. That is, if no term is a function of the independent variables alone. 
Otherwise, the equation is called nonhomogeneous or inhomogeneous. 


For example, the exponential growth equation, the wave equation, or the transport equation 
above are homogeneous. The mechanical vibrations equation above is nonhomogeneous as 
long as f(t) is not the zero function. Similarly, if the ambient temperature A is nonzero, 
Newton’s law of cooling is nonhomogeneous. A homogeneous linear ODE can be put into the 
form , 

an() 5 + Gn—1(2) — 


Compare to (3) and notice there is no function b(z). 


d 
eae: ai(«) + ag(x)y = 0. 


If the coefficients of a linear equation are actually constant functions, then the equation 
is said to have constant coefficients. The coefficients are the functions multiplying the 
dependent variable(s) or one of its derivatives, not the function b(z) standing alone. A 
constant coefficient nonhomogeneous ODE is an equation of the form 


dq” | qr} | dy | b( 
An T An— T - a - a = x 5 
da” ‘dxr! ‘dx 
where ao, @1,.--,@n are all constants, but b may depend on the independent variable x. The 


mechanical vibrations equation above is a constant coefficient nonhomogeneous second order 
ODE. The same nomenclature applies to PDEs, so the transport equation, heat equation 
and wave equation are all examples of constant coefficient linear PDEs. 


Finally, an equation (or system) is called autonomous if the equation does not explicitly 
depend on the independent variable. For autonomous ordinary differential equations, the 
independent variable is then thought of as time. Autonomous equation means an equation 
that does not change with time. For example, Newton’s law of cooling is autonomous, so is 
equation (5). On the other hand, mechanical vibrations or (4) are not autonomous. 


0.2.1 Exercises 


Exercise 0.2.1: Classify the following equations. Are they ODE or PDE? Is it an equation 
or a system? What is the order? Is it linear or nonlinear, and if it is linear, is it homogeneous, 
constant coefficient? If it is an ODE, is it autonomous? 
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pee ; du du 
a) sin(t) 7a + cos(t)z =t b) ae qt 7 =ay 
" " O7u O7u 
c) y +3y+5¢=0, x” +2-y=0 d) Fa + Uaa = 9 
pe 
" 2 
e) x +ta*°=t t) ag = 0 


Exercise 0.2.2:* Classify the following equations. Are they ODE or PDE? Is it an equation 
or a system? What is the order? Is it linear or nonlinear, and if it is linear, is it homogeneous, 
constant coefficient? If it is an ODE, is it autonomous? 


On Oo d 
a) pat + 35,0 = sin(e) b) = teste =O +41 
d'F 
c) —~ = 3F(z) d) y" + 8y' =1 
dx 
a) O7u 
e) ve’ +tyr’ =0, y’ +try=0 f) ae tY 
Exercise 0.2.3: If ti = (uy, U2, ug) is a vector, we have the divergence V - ti = Ou + = + ous 
and curl V x v= (33 — ae on _ Gus Guz — oa). Notice that curl of a vector is still a 


vector. Write out Maxwell’s equations in terms of partial derivatives and classify the system. 


Exercise 0.2.4: Suppose F is a linear function, that is, F(x,y) = ax + by for constants a 
and b. What is the classification of equations of the form Fy’, y) = 0. 


Exercise 0.2.5: Write down an explicit example of a third order, linear, nonconstant 
coefficient, nonautonomous, nonhomogeneous system of two ODE such that every derivative 
that could appear, does appear. 


Exercise 0.2.6:* Write down the general zeroth order linear ordinary differential equation. 
Write down the general solution. 


Exercise 0.2.7:* For which k is & +a" = t**? linear. Hint: there are two answers. 


Exercise 0.2.8: Write out an explicit example of a non-homogeneous fourth order, linear, 
constant coefficient differential equation. where all possible derivatives of the unknown 
function y appear. 


Exercise 0.2.9:* Let x, y, and z be three functions of t defined by the system of differential 
equations 
g=y yous 2 =38r-2Qyt+5z+e 


with initial conditions x(0) = 3, y(0) = —2 and z(0) = 1, and let u(t) be the function 
defined by the solution to 
u” — 5u” + 2u' — 3u=e' 


with initial conditions u(0) = 3, u’(0) = —2, and u"”(0) = 1. 


22 CHAPTER 0. INTRODUCTION 
a) Use the substitution u = x, u' = y, and u"” = z to verify that x(t) = u(t) because they 
solve the same initial value problem. 


b) What is the order of the system defining x, y, and z and how many components does it 
have? 


c) What is the order of the equation defining u? How many components does that have? 


d) How do these numbers relate to each other? 


Chapter 1 


First Order Differential Equations 


In this chapter, we begin by discussing first order differential equations. As they have the 
lowest possible order, only containing one derivative of the unknown function, they tend to 
be the simplest equations to try to analyze and solve. This doesn’t mean that we’ll be able to 
solve all of them, but we can make a decent effort at a fair number of them. These equations 
are also very common in modeling problems, as most principles from science and engineering 
give us a way to express the rate of change of a given quantity. This setup gives rise to a first 
order differential equation involving that quantity, which, if we can solve it, will tell us how 
the quantity evolves over time. Even if we can’t solve the equation analytically, a numerical 
solution may allow us to predict the behavior of a system over time and design it to best fit 
our needs. 


1.1 Integrals as solutions 


Attribution: [JL], §1.1. 


After this section, you will be able to: 


e Solve a first order differential equation by direct integration and 
e Understand the difference between a general solution and the solution to an initial 
value problem. 


A first order ODE is an equation of the form 


dy 

dr F(a, y); 
or just 

y' = f(x,y). 


Some examples that fit this form are 


/ . 
y =x2*y—e* sina 
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and 
y’ = e4(x* + 1) — cos(y). 


Looking back at the last section, the first of these is linear and the second is not. In general, 
there is no simple formula or procedure one can follow to find solutions. In the next few 
sections we will look at special cases where solutions are not difficult to obtain. In this section, 
let us assume that f is a function of x alone, that is, the equation is 


y' = f(z). (1.1) 


We could just integrate (antidifferentiate) both sides with respect to x. 


[v@e=ft@are 


that is 


Ws [ro ae 


This y(x) is actually the general solution. So to solve (1.1), we find some antiderivative of 
f(x) and then we add an arbitrary constant to get the general solution. 


Now is a good time to discuss a point about calculus notation and terminology. One 
of the final keystone concepts in Calculus 1 is that of the fundamental theorem of calculus, 
which ties together two mathematical ideas: definite integrals (defined as the area under a 
curve) and indefinite integrals or antidifferentiation (undoing the operation of differentiation). 
This theorem says that these two ideas are in some sense the same; in order to compute a 
definite integral, one can first find an antiderivative and plug in the endpoints (the most 
common use of the theorem), and that the derivative of a definite integral gives back the 
function inside (something that will be useful in this course). 

The main distinction between these two is the type of object that they are. Definite 
integrals evaluate to numbers, so they are functions, which means they are the object we 
want to deal with in this course. Indefinite integrals are families of functions, and while they 
have their uses (motivating the idea of a general solution), their main use is the process of 
antidifferentiation which leads us to solutions in the form of definite integrals. Provided that 
you can evaluate the antiderivative in question, these two processes will end up at exactly the 
same solution. If you end up confused about the terminology, the goal for this class is always 
a definite integral, but we can use antiderivatives to get there. Hence the terminology to 
integrate when we may really mean to antidifferentiate. Integration is just one way to compute 
the antiderivative (and it is a way that always works, see the following examples). Integration 
is defined as the area under the graph and it also happens to also compute antiderivatives. 
For sake of consistency, we will keep using the indefinite integral notation when we want an 
antiderivative, and you should always think of the definite integral as a way to write it. 


Example 1.1.1: Find the general solution of y’ = 32”. 


Solution: Elementary calculus tells us that the general solution must be y = 7? + C. Let us 
check by differentiating: y’ = 3x27. We got precisely our equation back. | 
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Normally, we will also have an initial condition such as y(xo) = yo for some two numbers 
xo and yo (Xo is often 0, but not always). If we do, the combination of a differential equation 
and an initial condition is called an initial value problem. We can then write the solution as 
a definite integral in a nice way. Suppose our problem is y/ = f(x), y(%o) = yo. Then the 
solution is 


y(2) = / Hvde tae (1.2) 


Let us check! We compute 


v(x) == Lf tte)as+ a]. 


Since Yo is a constant, it’s derivative is zero, and by the fundamental theorem of calculus 
| £6) ae =f) 
— s) de = F(a). 
dL Jag 


Therefore y’ = f(x), and by Jupiter, y is a solution. Is it the one satisfying the initial 
condition? Well, 


y(xo) = [ f(x) dx + yo 


and since f is a nice function, we know that the integral of f with matching endpoints is 0. 
Therefore y(xo) = yo. It is! 

Do note that the definite integral and the indefinite integral (antidifferentiation) are 
completely different beasts. The definite integral always evaluates to a number. Therefore, 
(1.2) is a formula we can plug into the calculator or a computer, and it will be happy to 
calculate specific values for us. We will easily be able to plot the solution and work with it 
just like with any other function. It is not so crucial to always find a closed form for the 
antiderivative. 


Example 1.1.2: Solve 
yao", y(0)=1. 


Solution: By the preceding discussion, the solution must be 


ya) = | eo geet, 
0 


Here is a good way to make fun of your friends taking second semester calculus. Tell them 
to find the closed form solution. Ha ha ha (bad math joke). It is not possible (in closed 
form). There is absolutely nothing wrong with writing the solution as a definite integral. 
This particular integral is in fact very important in statistics. | 


While there is nothing wrong with writing solutions as a definite integral, they should be 
simplified and evaluated if possible. Given the differential equation 
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the solution can be written as 7 
en) = | 3s” ds + 6. 
2 


However, it is much more convenient, both for human reasoning and computers, to write this 
solution as 
y(x) = 2° — 2. 


So, if integrals can be evaluated and simplified to explicit functions, then they should be 
worked out. If it is not possible, then answers in integral form are completely fine. 

Classical problems leading to differential equations solvable by integration are problems 
dealing with velocity, acceleration and distance. You have surely seen these problems before 
in your calculus class. 


Example 1.1.3: Suppose a car drives at a speed e‘/? meters per second, where t is time in 


seconds. How far did the car get in 2 seconds (starting at t = 0)? How far in 10 seconds? 


Solution: Let x denote the distance the car traveled. The equation is 


We just integrate this equation to get that 
a(t) = 2e/? +. 
We still need to figure out C. We know that when t = 0, then x = 0. That is, 7(0) = 0. So 
0 = 2(0) = 2e°/? + C =24+0C. 


Thus C = —2 and 
a(t) = 2e’/? — 2. 


Now we just plug in to get where the car is at 2 and at 10 seconds. We obtain 


a(2) = 2e?/? 23.44 meters, —_-x(10) = 2e!°/? — 2 = 294 meters. 


el 


Example 1.1.4: Suppose that the car accelerates at a rate of t?™/s?. At time t = 0 the car 
is at the 1 meter mark and is traveling at 10™/s. Where is the car at time t = 10? 


Solution: Well this is actually a second order problem. If x is the distance traveled, then 2’ 
is the velocity, and x” is the acceleration. The initial value problem for this situation is 


Van cae OS 1,. BO) S40. 


v =P, v(0) = 10. 
Once we solve for v, we can integrate and find x. _| 


Exercise 1.1.1: Solve for v, and then solve for x. Find x(10) to answer the question. 


1.1. INTEGRALS AS SOLUTIONS pig 


1.1.1 Exercises 
Exercise 1.1.2: Solve # = x? +x with y(1) =3. 


Exercise 1.1.3: Solve a = sin(52) with y(0) = 2. 


Exercise 1.1.4:* Solve % = e* + x with y(0) = 10. 


Exercise 1.1.5: Solve % = 2re** with y(0) = 1. 


Exercise 1.1.6: Solve @ = e! cos(2t) + t with y(0) =3. 


Exercise 1.1.7: Solve aw = hat + 3e” with y(0) = 2. 


Exercise 1.1.8: Solve wy = 4 for y(0) = 0. (This requires partial fractions or hyperbolic 


trigonometric functions.) 
Exercise 1.1.9 (harder): Solve y” = sinx for y(0) = 0, y/(0) = 2. 


Exercise 1.1.10: A spaceship is traveling at the speed 2t? + 1™/ (t is time in seconds). It 
is pointing directly away from earth and at time t = 0 it is 1000 kilometers from earth. How 
far from earth is it at one minute from time t = 0? 


Exercise 1.1.11:* Sid is in a car traveling at speed 10t + 70 miles per hour away from Las 
Vegas, where t is in hours. At t = 0, Sid is 10 miles away from Vegas. How far from Vegas is 
Sid 2 hours later? 


Exercise 1.1.12: Solve @ = sin(t?) +t, x(0) = 20. It is OK to leave your answer as a 
definite integral. 


Exercise 1.1.13: Solve % = e’ +sin(t), y(0) = 4. The answer can be left as a definite 
integral. 


Exercise 1.1.14: A dropped ball accelerates downwards at a constant rate 9.8 meters per 
second squared. Set up the differential equation for the height above ground h in meters. 
Then supposing h(0) = 100 meters, how long does it take for the ball to hit the ground. 


Exercise 1.1.15:* The rate of change of the volume of a snowball that is melting is 
proportional to the surface area of the snowball. Suppose the snowball is perfectly spherical. 
The volume (in centimeters cubed) of a ball of radius r centimeters is (4/3)mr?. The surface 
area is 4nr?. Set up the differential equation for how the radius r is changing. Then, suppose 
that at time t = 0 minutes, the radius is 10 centimeters. After 5 minutes, the radius is 8 
centimeters. At what time t will the snowball be completely melted? 


wy 


Exercise 1.1.16:* Find the general solution to y"" = 0. How many distinct constants do 


you need? 
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1.2 Slope fields 


Attribution: [JL], §1.2. 


After this section, you will be able to: 


e Identify or sketch a slope field for a first order differential equation and 


e Use the slope field to determine the trajectory of a solution to a differential 
equation. 


As we said, the general first order equation we are studying looks like 


y! = f(x,y). 


A lot of the time, we cannot simply solve these kinds of equations explicitly, because our 
direct integration method only works when the equation is of the form y’ = f(x), which we 
could integrate directly. In these more complicated cases, it would be nice if we could at least 
figure out the shape and behavior of the solutions, or find approximate solutions. 


1.2.1 Slope fields 


Suppose that we have a solution to the equation 
y! = f(a,y) with y(29) = yo. What does the fact | *| 7 7 | . 
that this solves the differential equation tell us about ; 
the solution? It tells us that the derivative of the i 
solution at this point will be f(x, yo). Graphically, 
the derivative gives the slope of the solution, so it — « ‘ 
means that the solution will pass through the point 
(Xo, Yo) and will have slope f(x, yo). For example, 
if f(x,y) = zy, then at point (2,1.5) we drawa | » 2 
short line of slope ry = 2 x 1.5 = 3. So, if y(x) isa 
solution and y(2) = 1.5, then the equation mandates 
that y’(2) = 3. See Figure 1.1. Figure 1.1: The slope y' = xy at 

To get an idea of how solutions behave, we draw | (2, 1.5). 
such lines at lots of points in the plane, not just the 
point (2, 1.5). We would ideally want to see the slope at every point, but that is just not 
possible. Usually we pick a grid of points fine enough so that it shows the behavior, but not 
too fine so that we can still recognize the individual lines. We call this picture the slope field 
of the equation. See Figure 1.2 on the next page for the slope field of the equation y/ = xy. 
Usually in practice, one does not do this by hand, but has a computer do the drawing. 

The idea of a slope field is that it tells us how the graph of the solution should be sloped, 
or should curve, if it passed through a given point. Having a wide variety of slopes plotted in 
our slope field gives an idea of how all of the solutions behave for a bunch of different initial 


% 
° 
nn 
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conditions. Which curve we want in particular, and where we should start the curve, depends 
on the initial condition. 

Suppose we are given a specific initial condition y(xo) = yo. A solution, that is, the graph 
of the solution, would be a curve that follows the slopes we drew, starting from the point 
(Xo, Yo). For a few sample solutions, see Figure 1.3. It is easy to roughly sketch (or at least 
imagine) possible solutions in the slope field, just from looking at the slope field itself. You 
simply sketch a line that roughly fits the little line segments and goes through your initial 
condition. The graph should “flow” along the little slopes that are on the slope field. 
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Figure 1.2: Slope field of y'! = xy. Figure 1.3: Slope field of y’ = xy with a 
graph of solutions satisfying y(0) = 0.2, 
y(0) =0, and y(0) = —0.2. 


By looking at the slope field we get a lot of information about the behavior of solutions 
without having to solve the equation. For example, in Figure 1.3 we see what the solutions 
do when the initial conditions are y(0) > 0, y(0) = 0 and y(0) < 0. A small change in the 
initial condition causes quite different behavior. We see this behavior just from the slope 
field and imagining what solutions ought to do. 

We see a different behavior for the equation y' = —y. The slope field and a few solutions 
is in see Figure 1.4 on the following page. If we think of moving from left to right (perhaps x 
is time and time is usually increasing), then we see that no matter what y(0) is, all solutions 
tend to zero as x tends to infinity. Again that behavior is clear from simply looking at the 
slope field itself. 


1.2.2. Exercises 


Exercise 1.2.1: Sketch slope field for y’ = e*~¥. How do the solutions behave as x grows? 
Can you guess a particular solution by looking at the slope field? 


Exercise 1.2.2:* Sketch the slope field of y’ = y>. Can you visually find the solution that 
satisfies y(0) = 0? 


Exercise 1.2.3: Sketch slope field for y' = x?. 
Exercise 1.2.4: Sketch slope field for y’ = y?. 
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Figure 1.4: Slope field of y' = —y with a graph of a few solutions. 


sketch out a slope field 


—3 < y <3 and determine the overall behavior of the solutions to the 
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For each of the following differential equations 


3<2<3 and 
equation as t — oo. If this fact depends on the value of the solution at t 
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explain how it 


? 


=0 


Which of the following slope fields corresponds to the differential equation 


t(y — 1). Explain your reasoning. 


Exercise 1.2.6 


dy _ 
dt 


Which of the following slope fields corresponds to the differential equation 


Exercise 1.2.7 


(2 — t)(y* — 9). Explain your reasoning. 


ol 


x(1 — y) to slope fields. 


Match equations y’ = 1— 12, y' = x — 2y, y/ 


1.2. SLOPE FIELDS 
Exercise 1.2.8 


Justify. 
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1, what will 


(3 — y)(y + 2) is below. 


If we find the solution to this differential equation with initial condition, y(0) 


The slope field for the differential equation 1 


happen to the solution as t + oo? Use the slope field and your knowledge of the equation to 


determine the long-time behavior of this solution. 


Exercise 1.2.12 
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Exercise 1.2.13: The slope field for the differential equation y' = (t — 2)(y + 4)(y — 3) is 
below. If we find the solution to this differential equation with initial condition, y(0) = 1, 
what will happen to the solution as t — co? Use the slope field and your knowledge of the 
equation to determine the long-time behavior of this solution. 


Exercise 1.2.14: The slope field for the differential equation y’ = (y + 1)(y +4) is below. 
If we find the solution to this differential equation with initial condition, y(0) = 1, what will 
happen to the solution as t + oo? Use the slope field and your knowledge of the equation to 
determine the long-time behavior of this solution. 


Figure 1.5: Exercise 1.2.12 Figure 1.6: Exercise 1.2.18 Figure 1.7: Exercise 1.2.14 


Exercise 1.2.15 (challenging): Take y’ = f(x,y), y(0) =0, where f(x,y) > 1 for all x and 
y. If the solution exists for all x, can you say what happens to y(x) as x goes to positive 
infinity? Explain. 


Exercise 1.2.16: Suppose y’ = f(x,y). What will the slope field look like, explain and 
sketch an example, if you know the following about f(x,y): 


a) f does not depend on y. b) f does not depend on x. 
c) f(t,t) =0 for any number t. d) f(¢.0) =0-and f(a 1) 1 forall 


Exercise 1.2.17: Describe what each of the following facts about the function f(x,y) tells 
you about the slope field for the differential equation y' = f(x,y). 


a) f(2,y) =0 for all y 


( 
Bb) {(¢;—2) =0 for all a 
G) J (ee) — 1 forall w 

( 


d) f(x,—-1) =0 for all x 
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1.3. Separable equations 


Attribution: [JL], §1.8. 


After this section, you will be able to: 


e Identify when a differential equation is separable, 


e Find the general solution of a separable differential equation, and 
e Solve initial value problems for separable differential equations. 


As mentioned in § 1.1, when a differential equation is of the form y' = f(a), we can just 
integrate: y = [ f(a) dx +C. Unfortunately this method no longer works for the general 
form of the equation y/ = f(x,y). Integrating both sides yields 


= | flevar+e 


Notice the dependence on y in the integral. Since y is a function of x, this expression is really 
of the form 


i ( f(x, y(a)) dx + C 


and without knowing what y(x) is in advance (which we don’t, because that’s what we are 
trying to solve for) we can’t compute this integral. Note that while you may have seen 
integrals of the form 


[teow dx 


in Calculus 3, this is not the same situation. In that class, x and y were both independent 
variables, so we could integrate this expression in x, treating y as a constant. However, here 
y is a function of x, so they are not both independent variables and y can not be treated like 
a constant. If y is a function of x and any y shows up in the integral, you can not compute it. 


1.3.1 Separable equations 


One particular type of differential equation that we can evaluate using a technique very 
similar to direct integration is separable equations. 


Definition 1.3.1 


We say a differential equation is separable if we can write it as 


for some functions f(x) and g(y). 
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Let us write the equation in the Leibniz notation 


dy 

H = f(a)aly) 
Then we rewrite the equation as 

dy 

Se = J (Ede 

g(y) ae 


It looks like we just separated the derivative as a fraction. The actual reasoning here is the 
differential from Calculus 1. This is the fact that for y a function of x, we know that 


dy 
dy = —dz. 
y dx . 


This means that we can take the equation 


4 = f(x)g(y), 


rearrange it as 


1 dy 
—_— = f(x 
gly) dx fle) 
and then multiply both sides by dx to get 
1 dy 
— — =f eax 
gly) dx fle) 


which leads to the rewritten equation above. Both sides look like something we can integrate. 


We obtain ‘ 
a eee 
(Ee [sears 


If we can find closed form expressions for these two integrals, we can, perhaps, solve for y. 


Example 1.3.1: Solve the equation 
y Say: 


Solution: Note that y = 0 is a solution. We will remember that fact and assume y 4 0 from 
now on, so that we can divide by y. Write the equation as oe = zy. Then 


[t= frase. 
¥y 


We compute the antiderivatives to get 


or 
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where D > 0 is some constant. Because y = 0 is also a solution and because of the absolute 
value we can write: 


22 
y= Der, 
for any number D (including zero or negative). 
We check: ’ : 
y = Durer =x (De® ) = ry 
Yay! | 


One particular case in which this method works very well is if the function f(z, y) is only 
a function of y. With this, we can explicitly complete the solution to equations like 


y’ = ky, 


reaching the solution y(x) = e**. 


We should be a little bit more careful with this method. You may be worried that we 
integrated in two different variables. We seemingly did a different operation to each side. Let 
us work through this method more rigorously. Take 


dy 


qa Fe) aly). 
We rewrite the equation as follows. Note that y = y(x) is a function of x and so is a 
1 dy 
—~- — = f(z). 
gly) dx ) 


We integrate both sides with respect to x: 


[aee- [toxre 


We use the change of variables formula (substitution) on the left hand side: 


1 
—d =i z)dz+C. 
lan y f@) 
And we are done. 


However, in some cases there are some special solutions to these problems as well that 
don’t fit the same formula. Assume we have 


dy 
Fe Se) aly) 

and we have a value yo such that g(yo) = 0. Then, the function y(x) = yo is a solution, 
provided f(a) is defined everywhere. (Plug this in and check!) This fills in the issue for 
having aa) in our integral expression, which is not defined when g(y) = 0. These are called 
singular solutions, and the next example will showcase one of them. 
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1.3.2 Implicit solutions 


We sometimes get stuck even if we can do the integration. Consider the separable equation 


We separate variables, 


We integrate to get 
y? we . 
= Se =a 

or perhaps the easier looking expression (where D = 2C’) 
y? + 2In|y| = 2? + D. 


It is not easy to find the solution explicitly as it is hard to solve for y. We, therefore, leave 
the solution in this form and call it an implicit solution. It is still easy to check that an 
implicit solution satisfies the differential equation. In this case, we differentiate with respect 
to x, and remember that y is a function of x, to get 


, 2 
y | 2y+—] = 22. 
y 


Multiply both sides by y and divide by 2(y? +1) and you will get exactly the differential 
equation. We leave this computation to the reader. 

If you have an implicit solution, and you want to compute values for y, you might have to 
be tricky. You might get multiple solutions y for each x, so you have to pick one. Sometimes 
you can graph x as a function of y, and then flip your paper. Sometimes you have to do more. 

Computers are also good at some of these tricks. More advanced mathematical software 
usually has some way of plotting solutions to implicit equations, which makes these solutions 
just as good for visualizing or graphing as explicit solutions. For example, for C' = 0 if you 
plot all the points (x,y) that are solutions to y? + 2In|y| = x, you find the two curves in 
Figure 1.8 on the next page. This is not quite a graph of a function. For each x there are two 
choices of y. To find a function you would have to pick one of these two curves. You pick the 
one that satisfies your initial condition if you have one. For example, the top curve satisfies 
the condition y(1) = 1. So for each C we really got two solutions. As you can see, computing 
values from an implicit solution can be somewhat tricky, but sometimes, an implicit solution 
is the best we can do. 

The equation above also has the solution y = 0. Since our function 
_ YY 
res 
is zero at y = 0, and gives an additional solution to the problem. The function y(x) = 0 
satisfies y'(a) = 0 and zu = 9 for all x, which is the right-hand side of the equation. So the 
general solution is 


g(y) 


y? + In |y| = 2? +C, and y=, 
These outlying solutions such as y = 0 are sometimes called singular solutions, as mentioned 
previously. 
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6.0 -5.0 
-5.0 -2.5 0.0 25 5.0 


Figure 1.8: The implicit solution y? + 2ln|y| = x? to y! = rae 


1.3.3 Examples of separable equations 


Example 1.3.2: Solve x7y/ = 1 — 2? + y? — xy’, y(1) =0. 
Solution: Factor the right-hand side 
w*y' = (1—2*)(1+y’). 


Separate variables, integrate, and solve for y: 


y’ Bee 
lt+y? a?’ 
yo 
i292 
+Yy £ 


—1 
arctan(y) = — —2+C, 
¢ 


-1 
y=tan (4-240). 
x 


Solve for the initial condition, 0 = tan(—2 + C) to get C= 2 (or C=2+4+7, or C= 24 27, 
etc.). The particular solution we seek is, therefore, 


—1 
y=tan (> -2 +2). 
a 
_ 


Example 1.3.3: Bob made a cup of coffee, and Bob likes to drink coffee only once reaches 
60 degrees Celsius and will not burn him. Initially at time t = 0 minutes, Bob measured the 
temperature and the coffee was 89 degrees Celsius. One minute later, Bob measured the 
coffee again and it had 85 degrees. The temperature of the room (the ambient temperature) 
is 22 degrees. When should Bob start drinking? 
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Solution: Let T be the temperature of the coffee in degrees Celsius, and let A be the ambient 
(room) temperature, also in degrees Celsius. Newton’s law of cooling states that the rate at 
which the temperature of the coffee is changing is proportional to the difference between the 


ambient temperature and the temperature of the coffee. That is, 
dT 
—=k(A-T 
= = A-T), 


for some constant k. For our setup A = 22, T(0) = 89, T'(1) = 85. We separate variables and 
integrate (let C and D denote arbitrary constants): 


1 dT 
S24 gS 
In(T — A) = —kt +C, (note that T — A > 0) 
T—-A=De™", 
T=A+De™. 


That is, T = 22 + De-**. We plug in the first condition: 89 = T(0) = 22 + D, and hence 
D = 67. So T = 22+67e-™. The second condition says 85 = T(1) = 22+67e~*. Solving for 
k we get k= —ln =~ 0.0616. Now we solve for the time ¢ that gives us a temperature of 
60 degrees. Namely, we solve 


60 = 22 + 67679090! 


60—22 
to get t = a ~ 9.21 minutes. So Bob can begin to drink the coffee at just over 9 


minutes from the time Bob made it. That is probably about the amount of time it took us 
to calculate how long it would take. See Figure 1.9. 


80 80 


60 60 


EN Cea a ee re roe or 


0.0 25 5.0 75 10.0 12.5 10) 20 40 60 80 


40 40 


Figure 1.9: Graphs of the coffee temperature function T(t). On the left, horizontal lines are 
drawn at temperatures 60, 85, and 89. Vertical lines are drawn at t= 1 and t = 9.21. Notice 
that the temperature of the coffee hits 85 att =1, and 60 at t + 9.21. On the right, the graph is 
over a longer period of time, with a horizontal line at the ambient temperature 22. 


= 
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Example 1.3.4: Find the general solution to y/ = =ay" (including singular solutions). 


Solution: First note that y = 0 is a solution (a singular solution). Now assume that y 4 0. 


aa, 
ye x, 
2 x 
pene aime 
yo 2.” 
_ 3 _ 6 
Y Pht 22+20° 
So the general solution is, 
6 
Y= ao and Y= 0 


Example 1.3.5: Find the general solution to 


dy 2 2 
—= 7 —3y-—4 
ob = (0? + e*)(y? — By — 4) 
Solution: Using the methods of separable equations, we can rewrite this differential equation 
as 4 
y — 2 x 
Mog. 8 ee 


and we can integrate both sides to solve. This leads to 


d 
[ote- [etee. 


The right-hand side of this can be integrated normally to give 
as 
[ere dv= +e +C 


and the left-hand side requires partial fractions in order to integrate correctly. If you are not 
familiar with this technique of partial fractions, it is reviewed in § B.3. 
Using the method of partial fractions, we want to rewrite 


1 A,B 
ye—38y—-4 y-4 yl 


and solve for A and B, which gives 


1 1s 15 
y—8y-4 y—-4 ysl 


Therefore, we can compute the integral 


dy 1/5 1/5 1 1 
A Ie da = dy = = In(|y — 4|) — <1 ToC, 
[ea [4 a 5 in(ly |) 5 in(ly + se 


40 CHAPTER 1. FIRST ORDER DIFFERENTIAL EQUATIONS 
Therefore, we can write the general solution as 


1 = 3 
n(t 1) =4 Let +0. 
5 \iytl 3 


We could solve this out for y as an explicit function, but that is not necessary for a problem 
like this. 


There are also two singular solutions here at y = 4 and y = —1. Notice that the implicit 
solution that we found previously is not defined at either of these values, because they involve 
taking the natural log of 0, which is not defined. = 


1.3.4 Exercises 
Exercise 1.3.1: Solve y' = y? for y(0) = 1. 
Exercise 1.3.2:* Solve x! = 4, x(1) =1. 


Exercise 1.3.3 (little harder): Solve y’ = (y—1)(y+1) for y(0) = 3. (Note: Requires 
partial fractions) 


Exercise 1.3.4:* Solve x’ = ECE z(0) = §. 


Exercise 1.3.5: Solve 4 = Fe for y(0) = 0. 
Exercise 1.3.6: Solve y' = */y. 
Exercise 1.3.7: Solve y' = x’y. 
Exercise 1.3.8:* Consider the differential equation 
dy _ 2 
dx y 
a) Find the general solution as an implicit function. 
b) Find the solution to this differential equation as an explicit function with y(1) = 4. 


c) Find the solution to this differential equation as an explicit function with y(0) = —2. 


Exercise 1.3.9:* Solve y’ = y", y(0) = 1, where n is a positive integer. Hint: You have to 
consider different cases. 


d 
Exercise 1.3.10: Solve 77 = (x? — 1)t, for x(0) =0. (Note: Requires partial fractions) 


d 
Exercise 1.3.11: Solve 77 =a-sin(t) for 2 (0) "1. 


Exercise 1.3.12:* Solve 4! = 2xry. 


Exercise 1.3.13: Solve y' = ye** with y(0) = 4. 
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Exercise 1.3.14: Solve 4 =azy+x+y+1. Hint: Factor the right-hand side. 
Exercise 1.3.15:* Solve 2! = 3xt? — 3t?, x(0) = 2. 

Exercise 1.3.16: Find the general solution of y’ = e*, and then y/' = e’. 
Exercise 1.3.17: Solve xy! = y+ 2x?y, where y(1) = 1. 


Exercise 1.3.18:* Find an implicit solution for x’ = 335, «(0) = 1. 


dy y?+1 
E ise 1.3.19: Solve — = f Si 
xercise olve 7 = aay for y(0) 
d 241 
Exercise 1.3.20: Find an implicit solution for oe lied for y(0) = 1. 
dx y?+1 
Exercise 1.3.21:* Find an implicit solution to y' = oy 


Exercise 1.3.22: Find an implicit solution for xy’ = as with y(3) = 2. 


Exercise 1.3.23: Find an explicit solution for y' = xe", y(0) = 1. 
Exercise 1.3.24:* Find an explicit solution to ry’ = y?, y(1) = 1. 
Exercise 1.3.25: Find an explicit solution for xy’ = e~", for y(1) = 1. 
Exercise 1.3.26: Find an explicit solution for y’ = y?(x* +1) with y(1) =2. 
Exercise 1.3.27: Find an explicit solution for y! = sont) with y(0) = 4. 


Exercise 1.3.28: Find an explicit solution for y! = ye" , y(0) = 1. It is alright to leave a 
definite integral in your answer. 


Exercise 1.3.29: Is the equation y’ = x + y+ 1 separable? If so, find the general solution, 
if not, explain why. 


Exercise 1.3.30: Is the equation y' = ty? +t separable? If so, find the general solution, if 
not, explain why. 


Exercise 1.3.31: Is the equation y' = xy? + 3y* — 4% — 12 separable? If so, find the general 
solution, if not, explain why. (Note: Requires partial fractions) 


Exercise 1.3.32: Suppose a cup of coffee is at 100 degrees Celsius at time t = 0, it is at 70 
degrees at t = 10 minutes, and it is at 50 degrees at t = 20 minutes. Compute the ambient 
temperature. 


Exercise 1.3.33:* Take Example 1.3.3 with the same numbers: 89 degrees at t = 0, 85 
degrees at t = 1, and ambient temperature of 22 degrees. Suppose these temperatures were 
measured with precision of +0.5 degrees. Given this imprecision, the time it takes the coffee 
to cool to (exactly) 60 degrees is also only known in a certain range. Find this range. Hint: 
Think about what kind of error makes the cooling time longer and what shorter. 
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Exercise 1.3.34:* A population x of rabbits on an island is modeled by x' = x — (1/1000) 2, 
where the independent variable is time in months. At time t = 0, there are 40 rabbits on the 


island. 
a) Find the solution to the equation with the initial condition. 


b) How many rabbits are on the island in 1 month, 5 months, 10 months, 15 months 
(round to the nearest integer). 


1.4. LINEAR EQUATIONS AND THE INTEGRATING FACTOR 43 


1.4 Linear equations and the integrating factor 


Attribution: [JL], §1.4. 


After this section, you will be able to: 


e Identify a linear first-order differential equation and write a first-order linear 
equation in standard form, 

e Solve initial value problems for first-order linear differential equations by inte- 
grating factors, and 

e Write solutions to first-order linear initial value problems in integral form if 
needed. 


One of the most important types of equations we will learn how to solve are the so-called 
linear equations. In fact, the majority of the course is about linear equations. In this section 
we focus on the first order linear equation. 


Definition 1.4.1 


A first order equation is linear if we can put it into the form: 


y' + p(x)y = f(z). 


The word “linear” means linear in y and y’; no higher powers nor functions of y or y’ 
appear. The dependence on x can be more complicated. 

Solutions of linear equations have nice properties. For example, the solution exists 
wherever p(x) and f(x) are defined, and has the same regularity (read: it is just as nice). 
We’ll see this in detail in § 1.5. But most importantly for us right now, there is a method for 
solving linear first order equations. In § 1.1, we saw that we could easily solve equations of 
the form 


dy _ 
apt) 


because we could directly integrate both sides of the equation, since the left hand side was 
the derivative of something (in this case, y) and the right side was only a function of x. We 
want to do the same here, but the something on the left will not be the derivative of just y. 

The trick is to rewrite the left-hand side of (1.3) as a derivative of a product of y with 
another function. Let r(x) be this other function, and we can compute, by the product rule, 


that 
d 


=, |r(@)y| = r@y! +r'@y. 


Now, if we multiply (1.3) by the function r(x) on both sides, we get 


r(x)y' + p(x)r(x)y = f(ax)r(2) 
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and the first term on the left here matches the first term from our product rule derivative. 
To make the second terms match up as well, we need that 


so that 
In |r| = f (2) dx 


or 
r(x) = ef P@) de, 


With this choice of r(x), we get that 


r(a)y + r(a)plady = + [r(a)u), 


so that if we multiply (1.3) by r(x), we obtain r(x)y’+r(x)p(x)y on the left-hand side, which 
we can simplify using our product rule derivative above to obtain 


£ [rtay] = r(0) £00) 


Now we integrate both sides. The right-hand side does not depend on y and the left-hand 
side is written as a derivative of a function. Afterwards, we solve for y. The function r(x) is 
called the integrating factor and the method is called the integrating factor method. 

This method works for any first order linear equation, no matter what p(x) and f(x) are. 
In general, we can compute: 


y+ plx)y = fl), 
el Pa) daa! 4 ef Pl) dn 7)y = ef P(e) de f(g) 


d 
ae jefn) iy] = oJ P(@) dz F(x), 


el Plo) dry — / ef PC) a F(a) da +C, 


y= e 7 (ff p00 c+) . 


Advice: Do not try to remember the formula itself, that is way too hard. It is easier to 
remember the process and repeat it. 

Of course, to get a closed form formula for y, we need to be able to find a closed form 
formula for the integrals appearing above. 


Example 1.4.1: Solve ; 
y +2ry =e", y(0) = —1. 
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Solution: First note that p(x) = 2x and f(x) = e*-*. The integrating factor is r(x) = 
ef p(x) dz — e** We multiply both sides of the equation by r(x) to get 


d 2 ‘ 
—le =e 
dx a 
We integrate 
2 
ey=e"4+C, 


y=e™ 40e™. 


Next, we solve for the initial condition —1 = y(0) = 1+C, so C = —2. The solution is 


x—x2 —a? 
y=e =e 


_ 


Note that we do not care which antiderivative we take when computing e/?@", You can 
always add a constant of integration, but those constants will not matter in the end. 


Exercise 1.4.1: Try it! Add a constant of integration to the integral in the integrating 
factor and show that the solution you get in the end is the same as what we got above. 


Since we cannot always evaluate the integrals in closed form, it is useful to know how 
to write the solution in definite integral form. A definite integral is something that you can 
plug into a computer or a calculator. Suppose we are given 


y +p(z)y= f(x), —-y(xo) = Yo. 


Look at the solution and write the integrals as definite integrals. 


ise cua 0 Jeg Pls) ds g f(t)dt-+y0) (1.4) 


You should be careful to properly use dummy variables here. If you now plug such a formula 
into a computer or a calculator, it will be happy to give you numerical answers. 


Exercise 1.4.2: Check that y(xo) = yo in formula (1.4). 


Example 1.4.2: Solve the initial value problem 
ty’ +4y=—t? —1 (1) =. 


Solution: In order to solve this equation, we want to put the equation in standard form, 
which is 

hae 4 ; 1 

ar ans 
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In this form, the coefficient p(t) of y is p(t) = . so that the integrating factor is 
r(t) — eS p(t) dt _ pa ee dt _ 4ln(t)_ 


Since 4 In(t) = In(t*), we have that r(t) = t*. Multiplying both sides of the equation by t* 
gives 

tty! “te At ?y = ra — £3 
where the left hand side is 4 (tty). Therefore, we can integrate both sides of the equation in 
t to give 


ty=——-—+C 
6 4 ¥ 
and we can solve out for y as 
(1) = i ” C 
WO 6 a ee 
To solve for C' using the initial condition, we plug in t = 1 to get that we need 
1 1 37 
Therefore, the solution to the initial value problem is 
@ 1 37/12 
) eee 


Example 1.4.3: Solve the initial value problem 
y +2ry=3 = y(0) =4. 


Solution: This equation is already in standard form. Since the coefficient of y is p(x) = 2z, 
we know that the integrating factor is 


re) = ef p@) de — 6? 
We can multiply both sides of the equation by this integrating factor to give 
y'e™ + 2Qre™ y = 3c” 


and then want to integrate both sides. The left-hand side of the equation is 4 [ey], so the 
antiderivative of that side is just yer. For the right-hand side, we would need to compute 


/ 3e"" Cx, 


which does not have a closed-form expression. Therefore, we need to represent this as a 
definite integral. Since our initial condition gives the value of y at zero, we want to use zero 
as the bottom limit of the integral. Therefore, we can write the solution as 


ye" = i 3e" ds +C 
0 
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and so can solve for y as 
Wey s al 86° deter. 
0 


Plugging in the initial condition gives that 
: 2 
y(0) =4= aA 3e° ds+Ce°=C. 
0 
Therefore, the solution to the initial value problem is 


Ue). oe, 3e° desde". 
0 
| 


Exercise 1.4.3: Write the solution of the following problem as a definite integral, but try 
to simplify as far as you can. You will not be able to find the solution in closed form. 


y ty=er, y(0) = 10. 


1.4.1 Exercises 


In the exercises, feel free to leave answer as a definite integral if a closed form solution cannot 
be found. If you can find a closed form solution, you should give that. 


Exercise 1.4.4: Solve y'+ ry = «x. 


Exercise 1.4.5: Solve y' + 6y = e”. 


Exercise 1.4.6: Solve y' + 4y = x7e~*. 


Exercise 1.4.7: Solve y' — 3y = xe’. 

Exercise 1.4.8: Solve y' + 3y = e* — e~** with y(0) = —3. 
Exercise 1.4.9: Solve y' — 2y =a +44. 
Exercise 1.4.10: Solve xy’ + 4y = x? — 5. 
Exercise 1.4.11: Solve ry’ — 3y = x — 2 with y(1) =3. 
Exercise 1.4.12: Solve y' — 4y = cos (3t). 

Exercise 1.4.13:* Solve y/ + 3x7y = 2”. 


Exercise 1.4.14: Solve y' + 3x?y = sin(x) e~®’, with y(0) = 1. 


Exercise 1.4.15: Solve y' + cos(x)y = cos(z). 
Exercise 1.4.16: Solve the IVP 4ty' + y = 24vt; y(10000) = 100. 


Exercise 1.4.17: Solve the IVP (t? + 1)y’ — 2ty=t? +1; y(1) =0. 
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Exercise 1.4.18: Solve wat y + zy = 3, with y(0) = 0. 
Exercise 1.4.19:* Solve y’ + 2sin(2x)y = 2sin(2z), y(*/2) = 3. 
Exercise 1.4.20: Consider the initial value problem 

5y’ — 3y =e 7 y(0) =a 


for an undetermined value a. Solve the problem and determine the dependence on the the 
value of a. How does the value of the solution as t — oo depend on the value of a? 


Exercise 1.4.21: Find an expression for the general solution to y' + 3y = sin(t?) with 
y(0) = 2. Simplfy your answer as much as possible. 
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1.5 Existence and Uniqueness of Solutions 


Attribution: [JL], §1.2. 


After this section, you will be able to: 


e Understand the terms existence and uniqueness as they apply to differential 


equations and 


e Find the maximum guaranteed interval of existence for the solution to an initial 
value problem. 


If we take the differential equation 


y' = f(x,y) y(Lo) = Yo, 
there are two main questions we want to answer about this equation. 
(a) Does a solution exist to the differential equation? 
(b) Is there only one solution to the differential equation? 


These are more commonly referred to as (a) existence of the solution and (b) uniqueness 
of the solution. These are especially crucial for equations that we are using to model a 
physical situation. For physical situations, the solution definitely exists (because the system 
does something and continues to exist) and the solution is unique, because a given system 
will always do the same thing given the same setup. Since we know that physical systems 
obey these properties, the equations we use to model them should have these properties as 
well. These properties do not necessarily hold for all differential equations, as shown in the 
examples below. 


Example 1.5.1: Attempt to solve: 


Integrate to find the general solution y = In |x| + C. The solution does not exist at x = 0. 
See Figure 1.10 on the next page. The equation may have been written as the seemingly 
harmless xy’ = 1. 


Example 1.5.2: Solve: 
y= 2y/lyl, —-y(0) = 0. 
See Figure 1.11 on the following page. Note that y = 0 is a solution. But another solution 


is the function 
os ft o> 0, 
y(z) = 


—x* if «<0. 
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Figure 1.10: Slope field of y' = 1/z. Figure 1.11: Slope field of y’ = 2,/|y| with two 
solutions satisfying y(0) = 0. 


What we see here is a significant problem for trying to represent physical situations. In 
the first there is no solution at x = 0, so if our physical scenario had wanted one, that would 
be an issue. Similarly, for the second, we do have solutions, but we have two of them, so we 
can’t use this to predict what is going to happen to a physical situation modeled by this 
equation over time. So, we need both existence and uniqueness to hold for our modeling 
equation in order to use differential equations to accurately model situations. Thankfully, 
these properties do apply to most equations, and we have fairly straight-forward criteria that 
can be used to determine if these properties are true for a given differential equation. For a 
first-order linear differential equation, the theorem is fairly straight-forward. 


Theorem 1.5.1 
Assume that we have the first-order linear differential equation given by 
y' + p(x)y = g(2). 


If p(x) and g(x) are continuous functions on an interval J that contains a point Zo, 
then for any y-value yo, the initial value problem 


y +p(z)y= g(t) —-y(Xo) = Yo 


has a unique solution. This solution exists and is unique on the entire interval /. 


The idea and proof of this theorem comes from the fact that we have an explicit method 
for solving these equations no matter what p and g are. We can always find an integrating 
factor for the equation, convert the left-hand side into a product rule term, take a definite 
integral of both sides, and then solve for y. Since we have this explicit formula, the solution 
will exist and be defined on the entire interval where the functions p and g are continuous. 
This also means that we can answer questions about where and for what values of x the 
solution to a differential equation exists. 
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Example 1.5.3: Consider the differential equation 
(x — 1)y'+ — e* 
Y es 5Y a 


What do the existence and uniqueness theorems say about the solution to this differential 
equation with the initial condition y(2) = 6? What about the solution with initial condition 
y(—3) = 1? 


Solution: To apply the existence and uniqueness theorem, we need to get the y/ term by 
itself. This results in the differential equation 
1 e” 


ee 


= 


In order to figure out where this solution exists and is unique, we need to determine where 
the coefficient functions p(x) and g(x) are continuous. The only two points that we have 
discontinuities are at x = 1 and x = 5. Therefore, if we have the initial condition y(2) = 6, 
we start at the x value of 2. Because this equation is linear, it will exist everywhere that 
these two functions are both continuous containing the point x = 2, and since the only 
discontinuities are at 1 and 5, we know that they are both continuous on (1,5). This means 
that we can take (1,5) as the interval J in the theorem, and know that this solution will exist 
and be unique on the interval (1, 5). 

For the other initial condition, y(—3) = 1, we now want an interval where these functions 
are continuous that contains —3. Again, we only have to avoid x = 1 and x = 5, so we can 
take the interval (—oo, 1) as the interval J in the theorem, and so we know the solution with 
this initial condition will exist and be unique on (—oo, 1). 

A convenient way to represent this situation is with a number line like that presented in 
Figure 1.12. On this number line, we mark the places where the functions p(x) or g(x) are 
discontinuous. 


y(2) =6 


y(-3) =1 


Figure 1.12: Number line representation of the existence intervals for a differential equation. 


To interpret this image, we can mark the initial point on the number line, where the point 
that we mark is the x coordinate of the initial condition. All of the intervals are in terms of 
x. Then, the existence and uniqueness theorem says that the solution will exist on the entire 
interval between any marked points on this number line. From that, we can see that the 
interval of existence for the initial condition y(2) = 6 is (1,5), and the interval for y(—3) = 1 


is (—oo, 1). a 
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For non-linear equations, we don’t have an explicit method of getting a solution that 
works for all equations. This means that we can’t fall back on this formula to guarantee 
existence or uniqueness of solutions. For this reason, we expect to get a result that is not 
as strong for non-linear equations. Thankfully, we do still get a result, which is known as 
Picard’s theorem‘. 


Theorem 1.5.2 (Picard’s theorem on existence and uniqueness) 


If f(x,y) is continuous (as a function of two variables) and = exists and is continuous 


near some (29, yo), then a solution to 


y! a (Ce, y), y(Zo) = Yo, 


exists (at least for some small interval of x’s) and is unique. 


The main fact that is “not as strong” about this result is the interval that we get from 
the theorem. For the linear theorem, we got existence and uniqueness on the entire interval 
I where p and g are continuous. For the non-linear theorem, we only get existence on 
some interval around the point x29. Even if f(x,y) and af are really nice functions that are 
continuous everywhere, we can still only guarantee existence on a small interval (that can 
depend on the initial condition) around the point Zo. 


Example 1.5.4: For some constant A, solve: 
y=y, yO) =A. 


Solution: We know how to solve this equation. First assume that A # 0, so y is not equal 
to zero at least for some x near 0. So a’ = V/y?, sox = —'/y+ C, soy = a If y(0) = A, 
then C = 1/4 so 


_ 1 
US Yaa 


If A = 0, then y = 0 is a solution. 
For example, when A = 1 the solution is 


which goes to infinity, and so “blows up”, at « = 1. This solution here exists only on the 
interval (—oo, 1), and hence, the solution does not exist for all « even if the equation is nice 
everywhere. The equation y’ = y? certainly looks nice. 

However, this fact does not contradict our existence and uniqueness theorem for non-linear 
equations. The theorem only guarantees that the solution to 


y=¥? 


exists and is unique on some interval containing 0. It does not guarantee that the solution 
exists everywhere that y? and its derivative are continuous, only that at each point where this 


*Named after the French mathematician Charles Emile Picard (1856-1941) 
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happens, the solution will exist for some interval around that point. The interval (—oo, 1) is 
“some interval containing 0”, so the theorem still applies and holds here. See the exercises for 
more detail on how this process works and how we can illustrate the fact that the interval of 
existence ise “some interval containing 0”. | 


The other main conclusion that we can draw from these theorems is the fact that two 
different solution curves to a first-order differential equation can not cross, provided the 
existence and uniqueness theorems hold. If y; and yz are two different solutions to y’ = f(z, y) 
and the solution curves for y;(x) and y2(x) cross, then this means that for some particular 
value of x7 and yo, we have that 


yi(%o) = Yo Yo(o) = Yo- 


If we pick Xp as a starting point, then the fact that the existence and uniqueness theorems 
hold imply that, at least for some interval around x9, there is exactly one solution to 


y =f(x,y) —-y(“o) = yo- 


However, both y; and ye satisfy these two properties. Therefore, y,; and ya must be the same, 
which doesn’t make sense because we assumed they were different. So it is impossible for 
two different solution curves to cross, provided the existence and uniqueness theorem holds. 
For a comparison, refer back to Example 1.5.2 earlier to see what non-uniqueness looks like, 
where we do have two solution curves that cross at the point (0,0). 

This fact is useful for analyzing differential equations in general, but will be particularly 
useful in § 1.7 in dealing with autonomous equations, where we can use simple solutions to 
provide boundaries over which other solutions can not cross. This fact will come up again in 
Chapters 4 and 5 in sketching trajectories for these solutions as well. 


Example 1.5.5: Consider the differential equation 


Y yay 4-4), 


1. Verify that y = 3 is a solution to this differential equation. 


2. Assume that we solve this problem with initial condition y(0) = 1. Is it possible for 
this solution to ever reach y = 4? Why or why not? 


Solution: 


1. If we take the function y(t) = 3, then y’ = 0, and plugging this into the right hand side 
also gives 0. Therefore, this function solves the differential equation. 


2. If the solutions starts with y(0) = 1, this means that it starts below the line y = 3. 
In order to get up to y = 4, the solution would need to cross over the line y = 3, 
which would mean that we have solution curves that cross. However, the function 
f(t,y) = (y — 3)°(y + 4) is continuous everywhere, as is the first derivative af = 
2(y — 3)(y +4) + (y — 3)”. Therefore, the existence and uniqueness theorem applies 
everywhere, and so solution curves can not cross. So, it is not possible for the solution 
to reach y = 4, because this would force solution curves to cross, which we know can 


not happen. ei 
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1.5.1 Exercises 
Exercise 1.5.1: Is it possible to solve the equation y' = ** for y(0) = 1? Justify. 


Exercise 1.5.2: Is it possible to solve the equation y! = y,/|x| for y(0) = 0? Is the solution 
unique? Justify. 


eli 
t+3" 


Exercise 1.5.3: Consider the differential equation y' + sy = 
a) Is this equation linear or non-linear? 
b) What is the maximum guaranteed interval of existence for the solution to this equation 
with initial condition y(0) = 3? 
c) What if we start with the initial condition y(4) = 0? 


Exercise 1.5.4: Consider the differential equation y'’ + ay = a. 


a) Is this equation linear or non-linear? 


b) What is the maximum guaranteed interval of existence for the solution to this equation 
with initial condition y(—3) = 1? 


c) What if we start with the initial condition y(2) = 5? 
d) What happens if we want to start with y(4) = 3? 

Exercise 1.5.5: Consider the differential equation (t + 3)y’ + t?y = 4. 
a) Is this equation linear or non-linear? 


b) What is the maximum guaranteed interval of existence for the solution to this equation 
with initial condition y(—2) = 1? 


c) What if we start with the initial condition y(—4) = 5? 
d) What happens if we want to start with y(4) = 2? 

Exercise 1.5.6: Consider the differential equation y' = y?. 
a) Is this equation linear or non-linear? 


b) What is the most we can say about the interval of existence for the solution to this 
equation with initial condition y(0) = 1? 


c) Find the solution to this differential equation with y(0) = 1. Over what values of x 
does this solution exist? 


d) Find the solution to this differential equation with y(0) = 4. Over what values of x 
does this solution exist? 


e) Find the solution to this differential equation with y(0) = —2. Over what values of x 
does this solution exist? 


f) Do any of these contradict your answer in (b)? 
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Exercise 1.5.7: Consider the differential equation y' = y? + 4. 
a) Is this equation linear or non-linear? 


b) What is the most we can say about the interval of existence for the solution to this 
equation with initial condition y(0) = 0? 


c) Find the solution to this differential equation with y(0) = 0. Over what values of x 
does this solution exist? 


Exercise 1.5.8: Consider the differential equation y' = x(y + 1)?. 
a) Is this equation linear or non-linear? 
b) If we set f(x,y) = x(y + 1)’, for what values of x and y are f and af continuous? 


c) What is the most we can say about the interval of existence for the solution to this 
equation with initial condition y(0) = 1? 


d) Find the solution to this differential equation with y(0) = 1. Over what values of x 
does this solution exist? 


Exercise 1.5.9 (challenging): Take (y — x)y’ = 0, y(0) =0. 

a) Find two distinct solutions. 

b) Explain why this does not violate Picard’s theorem. 
Exercise 1.5.10: Find a solution to y’ = |y|, y(0) = 0. Does Picard’s theorem apply? 
Exercise 1.5.11: Consider the IVP y' cost + ysint = 1; y(7/6) = 1. 


a) The Existence and Uniqueness Theorem guarantees a unique solution to this IVP on 
what interval? 


b) Find this solution explicitly. 


Exercise 1.5.12: Take an equation y' = (y — 2x)g(x,y) + 2 for some function g(x,y). Can 
you solve the problem for the initial condition y(0) = 0, and if so what is the solution? 


Exercise 1.5.13: Consider the differential equation y' = e*(y — 2). 
a) Verify that y = 2 is a solution to this differential equation. 


b) Assume that we look for the solution with y(0) = 0. Is it possible that y(«) = 3 for 
some later time x? Why or why not? 


c) Based on this, what do we know about the solution with y(0) = 5? 
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Exercise 1.5.14 (challenging): Suppose y’ = f(x,y) is such that f(x,1) = 0 for every x, f 
is continuous and af exists and is continuous for every x and y. 


a) Guess a solution given the initial condition y(0) = 1. 
b) Can graphs of two solutions of the equation for different initial conditions ever intersect? 


c) Given y(0) = 0, what can you say about the solution. In particular, can y(x) > 1 for 
any x? Can y(x) = 1 for any x? Why or why not? 


Exercise 1.5.15: Consider the differential equation y' = y? — 4. 
a) Verify that y = 2 and y = —2 are both solutions to this differential equation. 
b) Verify that the hypotheses of Picard’s theorem are satisfies for this equation. 


c) Assume that we solve this differential equation with y(0) = 1. Is it possible for the 
solution to reach y = 3 at any point? Why or why not? 


d) Assume that we solve this differential equation with y(0) = —1. Is it possible for the 
solution to reach y = —4 at any point? Why or why not? 


Exercise 1.5.16:* Is it possible to solve y' = xy for y(0) = 0? Is the solution unique? 
Exercise 1.5.17: Is it possible to solve y' = =*5 for y(1) = 0? 
Exercise 1.5.18 (tricky):* Suppose 


0 ify>Q0, 
w=" ify <0. 


Does y' = f(y), y(0) = 0 have a continuously differentiable solution? Does Picard apply? 
Why, or why not? 


Exercise 1.5.19:* Consider an equation of the form y' = f(x) for some continuous function 
f, and an initial condition y(xo) = yo. Does a solution exist for all x? Why or why not? 
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1.6 Numerical methods: Euler’s method 


Attribution: [JL], §1.7. 


After this section, you will be able to: 


e Use Euler’s method to numerically approximate solutions to first order differential 
equations, 
e Compute the error in a numerical method using the true solution, and 


e Compare a variety of numerical methods, including built-in Matlab methods. 


Unless f(x,y) is of a special form, it is generally very hard if not impossible to get a nice 
formula for the solution of the problem 


y=flz,y), yl) = 4. 


If the equation can be solved in closed form, we should do that. But what if we have an 
equation that cannot be solved in closed form? What if we want to find the value of the 
solution at some particular 7? Or perhaps we want to produce a graph of the solution to 
inspect the behavior. In this section we will learn about the basics of numerical approximation 
of solutions. 

The simplest method for approximating a solution is Fuler’s method*. It works as follows: 
Take xo and compute the slope k = f(2o, yo). The slope is the change in y per unit change 
in x. Follow the line for an interval of length h on the x-axis. Hence if y = yo at xg, then we 
say that y, (the approximate value of y at x; = 2%) +h) is y, = yo + hk. Rinse, repeat! Let 
k = f(#1,y1), and then compute ry = x, +h, and yo = y; + hk. Now compute x3 and y3 
using 22 and yo, etc. Consider the equation y’ = 9’/s, y(0) =1, and h = 1. Then 2p = 0 and 
yo = 1. We compute 


Ly =%t+h=0+4+1 = 1, Y1 = yo +h f (Xo, yo) = 14+1-/3 = 4/3 & 1.333, 
aps? 
t= 2%, +h=14+1=2, p= th flornmn) =Yot1- CD = v7 as 1.926. 
We then draw an approximate graph of the solution by connecting the points (29, yo), (1, 1), 
(2, Y2),.... For the first two steps of the method see Figure 1.13 on the following page. 
More abstractly, for any i = 0,1, 2,3,..., we compute 


te =a, Yinr = Ye th f (xi, yi). 


This can be worked out by hand for a few steps, but the formulas here lend themselves very 
well to being coded into a looping structure for more involved processes. The line segments 
we get are an approximate graph of the solution. Generally it is not exactly the solution. See 
Figure 1.14 on the next page for the plot of the real solution and the approximation. 


*Named after the Swiss mathematician Leonhard Paul Euler (1707-1783). The correct pronunciation of 
the name sounds more like “oiler.” 
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Figure 1.13: First two steps of Euler’s method with h =1 for the equation y/ = a with initial 
conditions y(0) = 1. 


3.0 
2.5 


2.0 


VALAWAASR ASSAY 
VULARR AOR AN 
LUNA KS RSMAS 
Wh A ADR SRR RS 
RR SOK Oe 
VANS Ui Ri Sob, 
Re Ne Si Se ee 

WO NS A as 
VLAN SK SSS 
WEA A gk, Sy at RO 
WN Oy A Os Kk AS SS 
ee ee 


0.5 


y 0.0 
cl 0 1 2 3 


Figure 1.14: Two steps of Euler’s method (step size 1) and the exact solution for the equation 


y = ve with initial conditions y(0) = 1. 


We continue with the equation y/ = »°/s, y(0) = 1. Let us try to approximate y(2) using 
Euler’s method. In Figures 1.13 and 1.14 we have graphically approximated y(2) with step 
size 1. With step size 1, we have y(2) & 1.926. The real answer is 3. We are approximately 
1.074 off. Let us halve the step size. Computing y4 with h = 0.5, we find that y(2) = 2.209, 
so an error of about 0.791. Table 1.1 on the facing page gives the values computed for various 
parameters. 


Exercise 1.6.1: Solve this equation exactly and show that y(2) = 3. 


The difference between the actual solution and the approximate solution is called the 
error. We usually talk about just the size of the error and we do not care much about its sign. 
The point is, we usually do not know the real solution, so we only have a vague understanding 
of the error. If we knew the error exactly ... what is the point of doing the approximation? 
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Error 
Previous error 


h Approximate y(2) Error 


1 1.92593 1.07407 
0.5 2.20861 0.79139 0.73681 
0.25 2.47250 0.52751 0.66656 
0.125 2.68034 0.31966 0.60599 
0.0625 2.82040 0.17960 0.56184 
0.03125 2.90412 0.09588 0.53385 
0.015625 2.95035 0.04965 0.51779 


0.0078125 2.97472 0.02528 0.50913 


Table 1.1: Euler’s method approximation of y(2) where of y! = 97/3, y(0) = 1. 


Notice that except for the first few times, every time we halved the step size the error 
approximately halved. This halving of the error is a general feature of Euler’s method as 
it is a first order method. There exists an improved Euler method, see the exercises, which 
is a second order method. A second order method reduces the error to approximately one 
quarter every time we halve the interval. The meaning of “second” order is the squaring in 
Ya = Yo x Ya = (/2)”. 

To get the error to be within 0.1 of the answer we had to already do 64 steps. To get it to 
within 0.01 we would have to halve another three or four times, meaning doing 512 to 1024 
steps. That is quite a bit to do by hand. The improved Euler method from the exercises 
should quarter the error every time we halve the interval, so we would have to approximately 
do half as many “halvings” to get the same error. This reduction can be a big deal. With 10 
halvings (starting at h = 1) we have 1024 steps, whereas with 5 halvings we only have to do 
32 steps, assuming that the error was comparable to start with. A computer may not care 
about this difference for a problem this simple, but suppose each step would take a second to 
compute (the function may be substantially more difficult to compute than »’/3). Then the 
difference is 32 seconds versus about 17 minutes. We are not being altogether fair, a second 
order method would probably double the time to do each step. Even so, it is 1 minute versus 
17 minutes. Next, suppose that we have to repeat such a calculation for different parameters 
a thousand times. You get the idea. 

Note that in practice we do not know how large the error is! How do we know what is 
the right step size? Well, essentially we keep halving the interval, and if we are lucky, we can 
estimate the error from a few of these calculations and the assumption that the error goes 
down by a factor of one half each time (if we are using standard Euler). 


Exercise 1.6.2: In the table above, suppose you do not know the error. Take the approximate 
values of the function in the last two lines, assume that the error goes down by a factor of 2. 
Can you estimate the error in the last time from this? Does it (approximately) agree with 
the table? Now do it for the first two rows. Does this agree with the table? 


Let us talk a little bit more about the example y’ = . y(0) = 1. Suppose that instead 
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of the value y(2) we wish to find y(3). The results of this effort are listed in Table 1.2 for 
successive halvings of h. What is going on here? Well, you should solve the equation exactly 
and you will notice that the solution does not exist at x = 3. In fact, the solution goes to 
infinity when you approach x = 3. 


h Approximate y(3) 


1 3.16232 

0.5 4.54329 

0.25 6.86079 
0.125 10.80321 
0.0625 17.59893 
0.03125 29.46004 
0.015625 50.40121 


0.0078125 87.75769 


Table 1.2: Attempts to use Euler’s to approximate y(3) where of y’ = ¥°/3, y(0) = 1. 


Another case where things go bad is if the solution oscillates wildly near some point. The 
solution may exist at all points, but even a much better numerical method than Euler would 
need an insanely small step size to approximate the solution with reasonable precision. And 
computers might not be able to easily handle such a small step size. 


In real applications we would not use a simple method such as Euler’s. The simplest 
method that would probably be used in a real application is the standard Runge-Kutta 
method (see exercises). That is a fourth order method, meaning that if we halve the interval, 
the error generally goes down by a factor of 16 (it is fourth order as 1/16 = 1/2 x 1/2 x 4/2 x 4/2). 

Choosing the right method to use and the right step size can be very tricky. There are 
several competing factors to consider. 


e Computational time: Each step takes computer time. Even if the function f is simple 
to compute, we do it many times over. Large step size means faster computation, but 
perhaps not the right precision. 


e Roundoff errors: Computers only compute with a certain number of significant digits. 
Errors introduced by rounding numbers off during our computations become noticeable 
when the step size becomes too small relative to the quantities we are working with. So 
reducing step size may in fact make errors worse. There is a certain optimum step size 
such that the precision increases as we approach it, but then starts getting worse as we 
make our step size smaller still. Trouble is: this optimum may be hard to find. 


e Stability: Certain equations may be numerically unstable. What may happen is that the 
numbers never seem to stabilize no matter how many times we halve the interval. We 
may need a ridiculously small interval size, which may not be practical due to roundoff 
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errors or computational time considerations. Such problems are sometimes called stiff. 
In the worst case, the numerical computations might be giving us bogus numbers that 
look like a correct answer. Just because the numbers seem to have stabilized after 
successive halving, does not mean that we must have the right answer. 


We have seen just the beginnings of the challenges that appear in real applications. 
Numerical approximation of solutions to differential equations is an active research area for 
engineers and mathematicians. For example, the general purpose method used for the ODE 
solver in Matlab and Octave (as of this writing) is a method that appeared in the literature 
only in the 1980s. 

The method used in Matlab and Octave is a fair bit different from the methods discussed 
previously. We don’t need to go too much in detail about it, but some information will 
be helpful. The main difference that will be visible when running these methods is that 
they are adaptive method. This means that they adjust the step-size based on what the 
differential equation looks like at a given point. Euler’s method, along with the improved 
Euler and Runge-Kutta methods, is a fixed step-size method, where the steps are always the 
same no matter what. Adaptive methods are harder to write and optimize, but can solve 
many problems faster because the adaptive nature of the method allows them to get similar 
accuracy to fixed step methods, but at many fewer steps. In the example below, the initial 
value problem 


dy 
—=S = 1 
ae y(0) 


is solved with an Euler’s method and Matlab’s built-in ode45 method. Both of the solutions 
are plotted along with the actual solution y = e! 


Solution 
r * Euler 
© ode45 


Figure 1.15: Comparison of the solution from Euler’s Method and ode45 to the actual solution 
dy _ 
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The Euler’s method takes 60 steps in this computation, but is still not as accurate as the 
ode45 method, which only takes 45 steps. In addition, the black diamonds, representing the 
different values computed by ode45 are not evenly spaced, illustrating the adaptive nature of 
this solver, while the red stars are all evenly spaced in the t-direction, as is expected from 
Euler’s method. 


1.6.1 Exercises 


d 
Exercise 1.6.3: Consider + = (2t— 2)’, x(0) = 2. Use Euler’s method with step size 
h = 0.5 to approximate x(1). 


Exercise 1.6.4: Consider the differential equation dy = ?t? —3y+1 with y(1) =4. Approx- 


imate the solution at t = 3 using Euler’s method with a step size of h = 1 and h = 0.5. 
Compare these values with the actual solution at t = 3. 


Exercise 1.6.5: Consider the differential equation wy = 2ty+y? with y(0) = 1. Approximate 
the solution at t = 2 using Euler’s method with a step size of h =1 and h = 0.5. 


d 
Exercise 1.6.6: Consider 77 =t—z, £(0) =1. 
a) Use Euler’s method with step sizes h = 1, 1/2, 1/4,1/s to approximate x(1). 


b) Solve the equation exactly. 


c) Describe what happens to the errors for each h you used. That is, find the factor by 
which the error changed each time you halved the interval. 


Exercise 1.6.7:* Let x’ = sin(xt), and x(0) = 1. Approximate x(1) using Euler’s method 
with step sizes 1, 0.5, 0.25. Use a calculator and compute up to 4 decimal digits. 


Exercise 1.6.8: Approximate the value of e by looking at the initial value problem y' = y 
with y(0) = 1 and approximating y(1) using Euler’s method with a step size of 0.2. 


Exercise 1.6.9:* Let x’ = 2t, and x(0) = 0. 
a) Approximate x(4) using Euler’s method with step sizes 4, 2, and 1. 
b) Solve exactly, and compute the errors. 
c) Compute the factor by which the errors changed. 
Exercise 1.6.10:* Let x’ = xe™*!, and x(0) = 0. 
a) Approximate x(4) using Euler’s method with step sizes 4, 2, and 1. 


b) Guess an exact solution based on part a) and compute the errors. 
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Exercise 1.6.11: Example of numerical instability: Take y' = —5y, y(0) = 1. We know that 
the solution should decay to zero as x grows. Using Euler’s method, start with h = 1 and 
compute Y1, Y2, y3, Y4 to try to approximate y(4). What happened? Now halve the interval. 
Keep halving the interval and approximating y(4) until the numbers you are getting start 
to stabilize (that is, until they start going towards zero). Note: You might want to use a 
calculator. 


There is a simple way to improve Euler’s method to make it a second order method by 
doing just one extra step. Consider a = f(x,y), y(%o) = yo, and a step size h. What we do 
is to pretend we compute the next step as in Euler, that is, we start with (2;,y;), we compute 
a slope k; = f(2;,y;), and then look at the point (x; + h,y; + kh). Instead of letting our 
new point be (x; +h, y; + kih), we compute the slope at that point, call it ky, and then take 
the average of k, and ko, hoping that the average is going to be closer to the actual slope on 
the interval from x; to x; +h. And we are correct, if we halve the step, the error should go 
down by a factor of 2? = 4. To summarize, the setup is the same as for regular Euler, except 
the computation of yj; and x41. 


ki = f (zany); Le = Cady 


ky + ko 
2 


h. 


ko = f(ti thy + kh), Yin = Wr 


Exercise 1.6.12:* Consider a =axz+y, y(0) =1. 
x 


a) Use the improved Euler’s method (see above) with step sizes h = 1/4 and h = 1/8 to 
approximate y(1). 

b) Use Euler’s method with h = 1/4 and h = 1/s. 

c) Solve exactly, find the exact value of y(1). 

d) Compute the errors, and the factors by which the errors changed. 


The simplest method used in practice is the Runge-Kutta method. Consider wy = Aba ee 
y(Zo) = Yo, and a step size h. Everything is the same as in Euler’s method, except the 
computation of yj4, and x41. 


ky = PERU s 
(2; 23 De, Yi ky (*/2)), Lig. = Ti+ h, 

ky + 2ko + 2kg +k 
(2; ye ae k(*/2)), Vit = Yt . ; sh 


i 
f - . 


ko = 
— 
kg = 


d 
Exercise 1.6.13: Consider =. Sr yu i. 
z 


a) Use Runge-Kutta (see above) with step sizes h = 1 and h = 1/2 to approximate y(1). 
b) Use Euler’s method with h = 1 and h = 1/2. 


c) Solve exactly, find the exact value of y(1), and compare. 
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1.7 Autonomous equations 


Attribution: [JL], §1.6. 


After this section, you will be able to: 


e Identify autonomous first order differential equations, 
e Find critical points or equilibrium solutions for autonomous equations, and 
e Sketch a phase line for these equations. 


Definition 1.7.1 


An equation of the form 
(eS 


where the derivative of solutions depends only on x (the dependent variable) is called 
an autonomous equation. If we think of t as time, the naming comes from the fact that 
the equation is independent of time. 


We return to the cooling coffee problem (Example 1.3.3). Newton’s law of cooling says 


dx 
a k(A—2), 

where x is the temperature, t is time, k is some positive constant, and A is the ambient 
temperature. See Figure 1.16 on the facing page for an example with k = 0.3 and A= 5. 

Note the solution x(t) = A (in the figure x = 5). We call these constant solutions the 
equilibrium solutions. The points on the z-axis where f(x) = 0 are called critical points of 
the differential equation (1.5). The point x = A is a critical point. In fact, each critical point 
corresponds to an equilibrium solution. 

Now, we want to determine what happens for other values of x that are not A. Based 
on the existence and uniqueness theorem in § 1.5 for first order differential equations, the 
fact that k(A — x) and its partial derivative in x, —k, are continuous everywhere gives that 
solution curves can not cross. This means that since we know x(t) = A is a solution, if a 
solution starts below z(t) = A, it must always stay there, and solutions that start above 
x(t) = A will also stay there. For more information about what the solutions do, we’ll need 
to look back at the equation and some sample solution curves. 

Note also, by looking at the graph, that the solution x = A is “stable” in that small 
perturbations in x do not lead to substantially different solutions as t grows. If we change 
the initial condition a little bit, then as t + oo we get x(t) > A. We call such a critical 
point asymptotically stable. In this simple example, it turns out that all solutions in fact go 
to A as t > oo. If there is a critical point where all nearby solutions move away from the 
critical point, we say it is unstable. If some nearby solutions go towards the critical point, 
and some others move away, then we say it is semistable. The final option is that solutions 
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nearby neither move towards nor away from the critical point, and these critical points are 
called stable. 

The last of these options may seem strange at first, and that is because stable critical 
points are not possible for autonomous equations with one unknown function. If a solution 
does not move towards or away from a critical point, that means it doesn’t move anywhere, 
and so is a critical point on its own. However, when we get to autonomous systems in § 4.7 
and § 5.1, we will see that in two dimensions, this is possible (think of a circle that does not 
spiral into or away from the center point). 
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Figure 1.16: The slope field and some solutions Figure 1.17: The slope field and some solutions 
of # =03(5—2). of z =0.12(5—2). 


Consider now the logistic equation 


« = kx(M — 2), 

for some positive k and M. This equation is commonly used to model population if we 
know the limiting population M/, that is the maximum sustainable population. The logistic 
equation leads to less catastrophic predictions on world population than x’ = kz. In the real 
world there is no such thing as negative population, but we will still consider negative x for 
the purposes of the math. 

See Figure 1.17 for an example, x’ = 0.12(5 — x). There are two critical points, x = 0 
and « = 5. The critical point at x = 5 is asymptotically stable, while the critical point at 
x = 0 is unstable. 

It is not necessary to find the exact solutions to talk about the long term behavior of the 
solutions. From the slope field above of x’ = 0.1”(5 — x), we see that 


5 if (0) > 0, 
lim x(t) = 40 i 20) =0, 
DNE or —co if x(0) <0. 
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Here DNE means “does not exist.” From just looking at the slope field we cannot quite 
decide what happens if x(0) < 0. It could be that the solution does not exist for ¢ all the way 
to oo. Think of the equation x’ = x”; we have seen that solutions only exist for some finite 
period of time. Same can happen here. In our example equation above it turns out that the 
solution does not exist for all time, but to see that we would have to solve the equation. In 
any case, the solution does go to —oo, but it may get there rather quickly. 

If we are interested only in the long term behavior of the solution, we would be doing 
unnecessary work if we solved the equation exactly. We could draw the slope field, but it is 
easier to just look at the phase diagram or phase line, which is a simple way to visualize the 
behavior of autonomous equations. The phase line for this equation is visible in Figure 1.18. 
In this case there is one dependent variable x. We draw the x-axis, we mark all the critical 
points, and then we draw arrows in between. Since x is the dependent variable we draw the 
axis vertically, as it appears in the slope field diagrams above. If f(x) > 0, we draw an up 
arrow. If f(a) < 0, we draw a down arrow. To figure this out, we could just plug in some x 
between the critical points, f(a) will have the same sign at all x between two critical points 
as long f(x) is continuous. For example, f(6) = —0.6 < 0, so f(x) < 0 for x > 5, and the 
arrow above x = 5 is a down arrow. Next, f(1) =0.4 > 0, so f(x) > 0 whenever 0< 2 <5, 
and the arrow points up. Finally, f(—1) = —0.6 < 0 so f(a) < 0 when xz < 0, and the arrow 
points down. 
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7 Sp 
A 
a) Set) 
v 


Figure 1.18: Phase line for the differential equation x’ = 0.14(5 — x). 


Armed with the phase diagram, it is easy to sketch the solutions approximately: As time 
t moves from left to right, the graph of a solution goes up if the arrow is up, and it goes 
down if the arrow is down. 


Exercise 1.7.1: Try sketching a few solutions simply from looking at the phase diagram. 
Check with the preceding graphs to see if you are getting the types of curves that match the 
solutions. 

Once we draw the phase diagram, we can use it to classify critical points as asymptotically 
stable, semistable, or unstable based on whether the “arrows” point into or away from the 
critical point on each side. Two arrows in means that the critical point is asymptotically 
stable, two arrows away means unstable, and one in one out means semistable. 


Example 1.7.1: Consider the autonomous differential equation 


dx 


ae = 2(x — 2)"(a + 3)(a — 4) (1.6) 


1.7. AUTONOMOUS EQUATIONS 67 


Find all equilibrium solutions for this equation, and determine their stability. Draw a phase 
line and use this information to sketch some approximate solution curves. 


Solution: This equation is already in factored form. This makes it simple to determine the 
equilibrium solutions as x = 0, x = 2, x = —3 and x = 4. In order to determine the stability 
of each critical point and draw the phase line, we need to plug in values surrounding these 
points to f(x) = x(x — 2)?(a + 3)(x — 4). We can see that 
—4)(—6)*(—1)(—8) <0, 
=1)(=3)" (2)(=5) > 0, 

(—1)*(4)(-3) < 0, 
f(3) = (3)(1)?(6)(-1) < 9, 

f(5) = (5)(3)°(8)(1) > 0. 

This lets us draw the phase line and determine the stability of each critical point. Thus, 
we see that x = —3 is an unstable critical point, c = 0 is asymptotically stable, « = 2 


is semistable, and x = 4 is unstable. A set of sample solution curves also validates these 
conclusions. 
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Figure 1.19: Phase line for the differential equation @ = x(x — 2)?(x + 3)(x — 4) and a plot of 
some solutions to this equation. 


=| 


1.7.1 Concavity of Solutions 


We can tell from the phase line for an autonomous equation when the solution will be 
increasing or decreasing. Is there any more we can learn about the shape of these graphs? 
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There is, and it comes from looking for the concavity, which is determined by the second 
derivative. 

We can compute the second derivative 

@ax  dydxr 

dt? dx El 
of our solution by noticing that oe = f(x). This function can be differentiated by the chain 
rule q ie 

at) = faa = f@)F@). 

So, the solution is concave up if f’(x) f(x) is positive, and concave down if that is negative. 
Phrased another way, the solution is concave up if f and f’ have the same sign, and it is 
concave down if f and f’ have opposite signs. 

Let’s see what this looks like in action. Take the logistic equation x’ = 0.14(5 — x), whose 
solutions are plotted in Figure 1.17. Figure 1.20 shows the graph of f(z) as a function of x 
for this scenario. When do f and f’ have the same sign? Well, this happens when f is both 
positive and increasing, or negative and decreasing. This happens between 0 and the vertex, 
as well as for x > 5. The vertex here is at x = 2.5, and so we conlude that the solution 
should be concave up when z is on the intervals (0, 2.5) and (5,00), and be concave down 
otherwise. Looking back at Figure 1.17, this is exactly what we observe. All of the solutions 
between 0 and 5 seem to “flip over” to be concave down when « crosses 2.5. 


Figure 1.20: ee of x us. f(x) for the differ- Figure 1.21: Plot of x vs. f(a) for the differ- 
ential equation G = 0.12(5 — 2). ential equation 42 = a(x — 2)?(x + 3)(a — 4). 


The same can be seen for solutions to (1.6), even though we can’t compute the extreme 
values explicitly. Figure 1.21 shows the graph of f(x) vs. x for this situation. Between each 
pair of equilibrium solutions there is a critical point of f (in the Calculus 1 sense) where 
the derivative is zero, and at this point, the derivative changes sign, and since the function 
value does not change sign, the concavity of the solution to the differential equation flips at 
this point. Comparing this graph and these points where concavity shifts with the solutions 
drawn in Figure 1.19 again validates these results. 
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1.7.2 Exercises 
Exercise 1.7.2: Consider x! = x”. 


a) Draw the phase diagram, find the critical points, and mark them asymptotically stable, 
semistable, or unstable. 


b) Sketch typical solutions of the equation. 
c) Find iim x(t) for the solution with the initial condition x(0) = —1. 
—0o 
Exercise 1.7.3: Consider x’ = sin x. 


a) Draw the phase diagram for —4a < x < 4m. On this interval mark the critical points 
asymptotically stable, semistable, or unstable. 


b) Sketch typical solutions of the equation. 


c) Find jim x(t) for the solution with the initial condition x(0) = 1. 
—> Co 


Exercise 1.7.4:* Let x! = (x —1)(x — 2)z?. 
a) Sketch the phase diagram and find critical points. 
b) Classify the critical points. 
c) If x(0) = 0.5, then find dim elt he 


Exercise 1.7.5: Let y’ = (y —2)(y?+1)(y+3). Sketch a phase diagram for this differential 
equation. Find and classify all critical points. If y(0) = 0, what will happen to the solution 
ast — oo? 


Exercise 1.7.6: Find and classify all equilibrium solutions for the differential equation 
x! = (x — 2)?(a@ + 1)(x@ + 3)3(a@ + 2). 


Exercise 1.7.7: Let y' = (y — 3)(y + 2)%e¥. Sketch a phase diagram for this differential 
equation. Find and classify all critical points. If y(0) = 0, what will happen to the solution 
ast — oo? 


d 
Exercise 1.7.8: Consider the DE a = y? —3y*+3y?—y?. Find and classify all equilibrium 


solutions of this DE. Then sketch a representative selection of solution curves. 
Exercise 1.7.9:* Let x’ =e™". 


a) Find and classify all critical points. b) Find him x(t) given any initial condition. 
00 


Exercise 1.7.10: Suppose f(x) is positive for 0 < x < 1, it is zero when x = 0 and x = 1, 
and it is negative for all other x. 
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a) Draw the phase diagram for x’ = f(x), find the critical points, and mark them 
asymptotically stable, semistable, or unstable. 


b) Sketch typical solutions of the equation. 


c) Find jim x(t) for the solution with the initial condition x(0) = 0.5. 
00 


Exercise 1.7.11:* Suppose @ = (x — a)(x — 8) for two numbers a < 8. 
a) Find the critical points, and classify them. 


For b), c), d), find kim x(t) based on the phase diagram. 
00 


b) 2(0) <a, c) @<a(0) <6; d) 8 < x(0). 


Exercise 1.7.12: A disease is spreading through the country. Let x be the number of 
people infected. Let the constant S be the number of people susceptible to infection. The 
infection rate a is proportional to the product of already infected people, x, and the number 
of susceptible but uninfected people, S — x. 


a) Write down the differential equation. 


b) Supposing x(0) > 0, that is, some people are infected at time t = 0, what is kim rea 
00 


c) Does the solution to part b) agree with your intuition? Why or why not? 
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1.8 Bifurcation diagrams 


Attribution: [JL], §1.6. 


After this section, you will be able to: 


e Draw and analyze bifurcation diagrams for autonomous equations with parameter. 


An extension of the topic of autonomous equation is autonomous equations with parameter. 
The idea is that we have a differential equation that has no explicit dependence on time, but 
does have a dependence on an outside parameter, which is a constant set by the physical 
situation. In terms of physical problems, this parameter will tend to be something that 
we can adjust to change how the differential equation behaves. For example, in a logistic 
differential equation 

“ = azr(K — 2) 
either the a or the K (or both) could be adjustable parameters. For a given value of the 
parameter, the differential equation behaves like a standard autonomous differential equation, 
but we can get different properties of this equation for different values of the parameter. 


Definition 1.8.1 


An autonomous equation with parameter qa is a differential equation of the form 


dx 


ae a Fa(z) 


where, for every value of a, fa(x) is a function of the single variable z. 


Later, we will want to view f(z) as a two-variable function of x and a, but for now, we 
want to think about it as a function of just x for a fixed value of a. We want to be able to 
analyze what happens to this equation for different values of a. Since it is an autonomous 
equation, we can do this using phase lines. This will be easiest to see through an example. 


Example 1.8.1: Consider the differential equation 


dx 

— = 2(x* —a), 

Em ol ) 
which fits the description of an autonomous equation with parameter a. Describe what 
happens in this differential equation for a = —4, a= 0, anda = 1. 
Solution: We can draw a phase line for a = —4,a =O anda=1._ It is clear that 
something happens with this equation between a = —4 and a = 1. We go from having 
only one equilibrium solution at a = —4 to having three equilibrium solutions at a = 1. In 
addition, the solution at y = 0 is unstable for a = —4, while it is asymptotically stable for 


a = 1. If we want to figure out when this change happens, we’ll need a better way to analyze 
this problem. 
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Figure 1.22: Phase lines for the differential equation a = a(z* — a) fora = —4, 0, 1. 


a 


How can we better approach this problem? The idea is to think about when the solution 
to the differential equation will be increasing or decreasing as a function of the two variables 
a and x. Based on the structure of the differential equation, the solution will be increasing 
when the function f,(a) is positive and will be decreasing when f,,(x) is negative. Since a 
phase line is a plot of this information for a given value of a, we essentially want to plot all 
of these phase lines on a two-dimensional graph. This graph is called a bifurcation diagram. 

dx 


Figure 1.23 shows a bifurcation diagram for the example F = x(x? — a). 


Figure 1.23: Bifurcation Diagram for the differential equation a = a(a?— a). In this figure, a 


blue region means the solution will be increasing and red indicates decreasing. 
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Within this picture, we can see all of our phase lines from before, because at any value of 
a, taking the vertical slice of this graph at that value, we get the phase line. If we want to 
consider a = —4, then we can look above the horizontal coordinate —4, and that will give us 
the phase line for a = —4. The same goes for any other value of @ we want to consider. For 
instance, we can also see that for any a < 0, there will be one equilibrium solution, and for 
a > 0 there are three equilibrium solutions, indicated by the three black curves above each of 
those a values. 

From this, we can see that the point at which the behavior changes is a = 0. Thus, for 
this problem a = 0 is called the bifurcation point. This is defined to be the value of the 
parameter for which the overall behavior of the equation changes. This can be a change in 
the number of equilibrium solutions, the stability of these equilibrium solutions, or both. 
For this particular example, we have both of these. We go from 1 equilibrium solution to 3, 
and the solution at y = 0 changes in stability. This type of bifurcation is called a “pitchfork 
bifurcation” based on the shape of the equilibrium solutions near the bifurcation point. 

Another example of a bifurcation of a different form can be seen in the example of the 
logistic equation with harvesting. Suppose an alien race really likes to eat humans. They 
keep a planet with humans on it and harvest the humans at a rate of h million humans per 
year. Suppose x is the number of humans in millions on the planet and ¢ is time in years. Let 
M be the limiting population when no harvesting is done. The number k > 0 is a constant 
depending on how fast humans multiply. Our equation becomes 


d 
oa = kz(M —2)—h. 
In this setup, M and k are fixed values, and the parameter that is being adjusted for this 
equation is h. We expand the right-hand side and set it to zero. 
ka(M — x) -—h=—kz? +kMz—h=0. 


Solving for the critical points using the quadratic formula, let us call them A and B, we get 


P kM + 4/(kM)* — 4hk kM —4/(kM)* — 4hk 
7 2k 7 2k ' 
Exercise 1.8.1: Sketch a phase diagram for different possibilities. Note that these possibili- 


ties are A > B, or A= B, or A and B both complex (i.e. no real solutions). Hint: Fix some 
simple k and M and then vary h. 


Example 1.8.2: For example, let M = 8 and k = 0.1. What happens for different values of 
h in this situation? 


Solution: When h = 1, then A and B are distinct and positive. The slope field we get is 
in Figure 1.24 on the following page. As long as the population starts above B, which is 
approximately 1.55 million, then the population will not die out. It will in fact tend towards 
A & 6.45 million. If ever some catastrophe happens and the population drops below B, 
humans will die out, and the fast food restaurant serving them will go out of business. 
When h = 1.6, then A = B =4. There is only one critical point and it is semistable. 
When the population starts above 4 million it will tend towards 4 million. If it ever drops 
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Figure 1.24: The slope field and some solutions Figure 1.25: The slope field and some solutions 
of x’ =0.1x2(8-—2)-1. of x’ =0.12(8—2x)—-1.6. 


below 4 million, humans will die out on the planet. This scenario is not one that we (as the 
human fast food proprietor) want to be in. A small perturbation of the equilibrium state and 
we are out of business. There is no room for error. See Figure 1.25. 

Finally if we are harvesting at 2 million humans per year, there are no critical points. 
The population will always plummet towards zero, no matter how well stocked the planet 
starts. See Figure 1.26. 
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Figure 1.26: The slope field and some solutions of x’ = 0.1a(8— 2x) —2. 


All of these can also be seen from the bifurcation diagram, which is drawn in Figure 1.27 
on the next page. The values A and B discussed above represent the upper and lower branches 
of the parabola in the figure. For any h > 1.6, there are no equilibrium solutions and the 
phase line is entirely decreasing, meaning the solution will converge to zero no matter what. 
For h < 1.6, there are two equilibrium solutions, with the top one asymptotically stable and 
the bottom one unstable. At h = 1.6 is where the bifurcation point occurs for this example. 
This is an example of a “saddle-node” bifurcation, as the two equilibrium solutions collide 
with each other at the bifurcation point and disappear. 
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Figure 1.27: Bifurcation diagram for the differential equation x’ = 0.1a4(8—2x)—h. 


Another way to visualize this situation is by plotting the function f,(x) for the different 
values of a. The places where this function is zero give the equilibrium solutions, and we can 
determine bifurcation values by looking for where the zeros of this function change behavior. 
For this particular example, the graphs of f,(2) are drawn in Figure 1.28. 


Figure 1.28: Graph of fa(x) = 0.14(8 — x) —a fora =0, 1.0, 1.6, 2.0. 


a 


The values of a we are looking for are those where the number and types of zeros change 
for the function f(x). In this figure, we see that for a < 1.6, the parabola crosses the x 
axis twice, resulting in two zeros and two equilibrium solutions. For a = 1.6, there is one 
(double) root, and for a > 1.6, there are no equilibrium solutions, and the function f,(z) is 
always negative. Since the number of roots/zeros changes at a = 1.6, that means that 1.6 
is the bifurcation point for this equation. We can also see this from the equation, since the 
equilibrium solutions are determined by the values of x where 


0.12(8 — x) -a=0 or — 0.12” + 0.82 —a=0 
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which can be found by the quadratic formula 


_ 0.8 + 1/0.64 — 4(0.1)(a) 
an 0.2 


Roots to this equation do not exist (because they are complex) if 0.64 —0.4a < 0, or a > 1.6. 


1.8.1 Exercises 


Exercise 1.8.2: Start with the logistic equation oe = kx(M — x). Suppose we modify our 


harvesting. That is we will only harvest an amount proportional to current population. In 
other words, we harvest hx per unit of time for some h > 0 (Similar to earlier example with 
h replaced with ha). 


a) Construct the differential equation. 
b) Show that ifkM > h, then the equation is still logistic. 
c) What happens when kM < h? 


Exercise 1.8.3:* Assume that a population of fish in a lake satisfies & = kx(M — x). Now 
suppose that fish are continually added at A fish per unit of time. 


a) Find the differential equation for x. b) What is the new limiting population? 


Exercise 1.8.4: Consider the differential equation with parameter a given by y' = y(y — 
a+1). 


a) Sketch a phase diagram for this differential equation with a = —3, a= 1, anda =3. 
b) Draw a bifurcation diagram for this differential equation with parameter. 


c) What is the bifurcation point for this equation? What changes when a passes over the 
bifurcation point? 


Exercise 1.8.5: Consider the differential equation with parameter a given by y' = y*(y?—a). 
a) Sketch a phase diagram for this differential equation with a = —3, a = 0, anda = 3. 
b) Draw a bifurcation diagram for this differential equation with parameter. 


c) What is the bifurcation point for this equation? What changes when a passes over the 
bifurcation point? 


Exercise 1.8.6: Consider the differential equation with parameter a given by y’ = y(a—y). 
a) Sketch a phase diagram for this differential equation with a = —3, a = 0, anda = 3. 
b) Draw a bifurcation diagram for this differential equation with parameter. 


c) What is the bifurcation point for this equation? What changes when a passes over the 
bifurcation point? 
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1.9 Exact equations 


Attribution: [JL], §1.8. 


After this section, you will be able to: 


Determine if a first order differential equation is exact, 
Find the general solution to an exact equation, 
Solve initial value problems for exact equations, and 


Use integrating factors to make some non-exact equations exact in order to solve 
them. 


Another type of equation that comes up quite often in physics and engineering is an 
exact equation. Suppose F(x, y) is a function of two variables, which we call the potential 
function. The naming should suggest potential energy, or electric potential. Exact equations 
and potential functions appear when there is a conservation law at play, such as conservation 
of energy. Let us make up a simple example. Let 


F(a,y)=2°+y’. 


We are interested in the lines of constant 
energy, that is lines where the energy is con- 
served; we want curves where F(z,y) = C, 
for some constant C’, since F’ represents the 
energy of the system. In our example, the 


curves x? + y* = C are circles. See Fig- 
ure 1.29. 0 0 
We take the total derivative of F: 
OF OF z s 
dF = —dz + —dy. 

Ox Oy 7 
For convenience, we will make use of the | ™ 7 + P . o ° 
notation of Ff, = a and Fy = In our | Figure 1.29: Solutions to F(a,y) = x? +y? = 
example, C for various C. 


dF = 2¢ dx + 2y dy. 


We apply the total derivative to F(x, y) = C, to find the differential equation dF’ = 0. The 
differential equation we obtain in such a way has the form 
dy 


Mdz+Ndy =0, or a ae 
a 
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Definition 1.9.1 


An equation of the form 
dy 


M(a,y) + N(a, y) oe =) 


is called exact if it was obtained as dF = 0 for some potential function F’. 


In our simple example, we obtain the equation 


d 
2a dx + 2ydy = 0, or 2n + 2y = =0. 
ia 


Since we obtained this equation by differentiating x? + y? = C,, the equation is exact. We 
often wish to solve for y in terms of x. In our example, 


y = +VC? — 2?. 


An interpretation of the setup is that at each point in the plane v = (M,N) is a vector, 
that is, a direction and a magnitude. As M and N are functions of (x, y), we have a vector field. 
The particular vector field v that comes from an exact equation is a so-called conservative 
vector field, that is, a vector field that comes with a potential function F(x, y), such that 


. OF OF 
v= | —,— }. 
Ox’ Oy 
This is something that you may have seen in your Calculus 3 course, and if so, the process 
for solving exact equations is basically identical to the process of finding a potential function 
for a conservative vector field. The physical interpretation of conservative vector fields is as 


follows. Let y be a path in the plane starting at (x1, y1) and ending at (x2, y2). If we think 
of v as force, then the work required to move along ¥ is 


[a ar= [Mae Ndy = Pleay) - Fenn). 
y y 


That is, the work done only depends on endpoints, that is where we start and where we end. 
For example, suppose F' is gravitational potential. The derivative of F given by v is the 
gravitational force. What we are saying is that the work required to move a heavy box from 
the ground floor to the roof only depends on the change in potential energy. That is, the work 
done is the same no matter what path we took; if we took the stairs or the elevator. Although 
if we took the elevator, the elevator is doing the work for us. The curves F'(x,y) = C are 
those where no work need be done, such as the heavy box sliding along without accelerating 
or breaking on a perfectly flat roof, on a cart with incredibly well oiled wheels. Effectively, 
an exact equation is a conservative vector field, and the implicit solution of this equation is 
the potential function. 


1.9.1 Solving exact equations 


Now you, the reader, should ask: Where did we solve a differential equation? Well, in 
applications we generally know M and JN, but we do not know F’. That is, we may have just 
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started with 2x + 2y% = 0), or perhaps even 


d 
r+yp =0. 


It is up to us to find some potential F that works. Many different F' will work; adding 
a constant to F' does not change the equation. Once we have a potential function F', the 
equation F(x, y(x)) = C gives an implicit solution of the ODE. 


Example 1.9.1: Let us find the general solution to 2x + 2yH = 0. Forget we knew what F 
was. 


Solution: If we know that this is an exact equation, we start looking for a potential function 
F. We have M = 2x and N = 2y. If F exists, it must be such that F(x, y) = 2x. Integrate 
in the x variable to find 

F(x,y) = 2? + Aly), (1.7) 


for some function A(y). The function A is the “constant of integration”, though it is only 
constant as far as x is concerned, and may still depend on y. Now differentiate (1.7) in y and 
set it equal to NV, which is what F, is supposed to be: 


2y = F,(x,y) = A'(y). 


Integrating, we find A(y) = y?. We could add a constant of integration if we wanted to, but 
there is no need. We found F(z,y) = 2? + y”. Next for a constant C, we solve 


F(ax,y(«)) =. 
for y in terms of x. In this case, we obtain y = +C? — x? as we did before. _| 


Exercise 1.9.1: Why did we not need to add a constant of integration when integrating 
A'(y) = 2y? Add a constant of integration, say 3, and see what F you get. What is the 
difference from what we got above, and why does it not matter? 


In the previous example, you may have also noticed that the equation 2x” + 2yZ = (is 
separable, and we could have solved it via that method as well. This is not a coincidence, 
as every separable equation is exact (see Exercise 1.9.15 for the details) but there are many 
exact equations that are not separable, which we will see throughout the examples here. 

The procedure, once we know that the equation is exact, is: 


(i) Integrate F, = M in « resulting in F(x, y) = something + A(y). 


(ii) Differentiate this F in y, and set that equal to N, so that we may find A(y) by 
integration. 


The procedure can also be done by first integrating in y and then differentiating in x. Pretty 
easy huh? Let’s try this again. 


Example 1.9.2: Consider now 2x + y + cy =, 
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Solution: OK, so M = 27+ y and N = xy. We try to proceed as before. Suppose F’ exists. 
Then F;,(z,y) = 22+ y. We integrate: 


F(2,y) = 2° + cy + Aly) 
for some function A(y). Differentiate in y and set equal to N: 
N= sy = F,(z,y) = 2+ A'(y). 


But there is no way to satisfy this requirement! The function xy cannot be written as x plus 
a function of y. The equation is not exact; no potential function F’ exists. _| 


Is there an easier way to check for the existence of F’, other than failing in trying to find 
it? Turns out there is. Suppose M = F,, and N = F,. Then as long as the second derivatives 
are continuous, 

OM OF OF _ aN 
Oy  OyOx Oxdy Ox” 


Let us state it as a theorem. Usually this is called the Poincaré Lemma’. 


Theorem 1.9.1 (Poincaré) 


If M and N are continuously differentiable functions of (x, y), and a = oN then near 


any point there is a function F(x, y) such that M = oF amc) Ve = 


The theorem doesn’t give us a global F’ defined everywhere. In general, we can only find 
the potential locally, near some initial point. By this time, we have come to expect this from 
differential equations. 

Let us return to the example above where M = 2x + y and N = zy. Notice M, = 1 and 
N, = y, which are clearly not equal. The equation is not exact. 


Example 1.9.3: Solve 


dy —2x-y 
= 0) =1. 
Solution: We write the equation as 
dy 
2 —1)—=0 
(x+y) +(e-) 4 =0, 
so M=2xr+y and N=a2-—1. Then 
My = l= Ne 


The equation is exact. Integrating M in x, we find 


F(a.y) = a? + cyt A(y). 


*Named for the French polymath Jules Henri Poincaré (1854-1912). 
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Differentiating in y and setting to N, we find 
xr-1l=2+4+A"(y). 


So A’(y) = —1, and A(y) = —y will work. Take F(z,y) = 2? + xy — y. We wish to 
solve x7 + 2y —y = C. First let us find C. As y(0) = 1 then F(0,1) = C. Therefore 
0?+0x1—1=C,so C =-1. Now we solve x? + zy — y = —1 for y to get 


_ Sad 
a = 


Example 1.9.4: Solve 


x 
1 dr + 


—Byepetayppya9 yd) =2. 


Solution: We leave to the reader to check that M, = Nz. 

This vector field (IM, NV) is not conservative if considered as a vector field of the entire 
plane minus the origin. The problem is that if the curve ¥ is a circle around the origin, say 
starting at (1,0) and ending at (1,0) going counterclockwise, then if F’ existed we would 
expect 


0 = F(1,0) — F(1,0) = | F,dx+F,dy = dix + dy = 2n. 
i | OCD [ape ae 


That is nonsense! We leave the computation of the path integral to the interested reader, or 
you can consult your multivariable calculus textbook. So there is no potential function F’ 
defined everywhere outside the origin (0,0). 

If we think back to the theorem, it does not guarantee such a function anyway. It only 
guarantees a potential function locally, that is only in some region near the initial point. As 
y(1) = 2 we start at the point (1,2). Considering x > 0 and integrating M in x or N in y, 
we find 

F (x,y) = arctan(¥/z). 
The implicit solution is arctan(¥/*) = C. Solving, y = tan(C)x. That is, the solution is a 


straight line. Solving y(1) = 2 gives us that tan(C’) = 2, and so y = 2z is the desired solution. 
See Figure 1.30 on the following page, and note that the solution only exists forr>0. — | 


Example 1.9.5: Solve 
dy 
dx 
Solution: The reader should check that this equation is exact. Let M = x? + y? and 
N = 2y(a +1). We follow the procedure for exact equations 


x? + y? + 2y(2+1)— =0. 


i 
F(z,y) = ge + xy” + A(y), 


and 
2y(a +1) = 2ry + A’(y). 
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Figure 1.30: Solution to rane dx + Pig dy = 0, y(1) = 2, with initial point marked. 


Therefore A’(y) = 2y or A(y) = y? and F(z, y) = qa +xy?+y?. We try to solve F(z, y) = C. 
We easily solve for y? and then just take the square root: 


C-—(1 3 C—(1 3 
paca, (C= 
r+ ae ap 
When x = —1, the term in front of oe vanishes. You can also see that our solution is not 


valid in that case. However, one could in that case try to solve for x in terms of y starting 
from the implicit solution a3 + xy? + y? =C. The solution is somewhat messy and we leave 
it as implicit. = 


1.9.2 Integrating factors 


Sometimes an equation M dz+ N dy = 0 is not exact, but it can be made exact by multiplying 
with a function u(z,y). That is, perhaps for some nonzero function u(z, y), 


is exact. Any solution to this new equation is also a solution to M dx + N dy = 0. 
In fact, a linear equation 


Ys p(x)y = f(a), or ~— (pla) — f(w)) de + dy = 0 


is always such an equation. Let r(x) = e/ ?() 4 be the integrating factor for a linear equation. 
Multiply the equation by r(x) and write it in the form of M+ N# =i); 


dy 
r(x)p(x)y — r(a) f(a) + rays” = 0. 
Then M = r(x)p(x)y—r(a) f(x), so M, = r(x)p(x), while N = r(x), so Nz = r'(x) = r(x)p(a). 
In other words, we have an exact equation. Integrating factors for linear functions are just a 
special case of integrating factors for exact equations. 
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But how do we find the integrating factor u? Well, given an equation 
Mdx+Ndy=0, 


u should be a function such that 


O O 
By eM] = uM +uM, = 5, UN] =u,N +uNy,. 
Therefore, 

(M, — Nz)u = u,N — uM. 


At first it may seem we replaced one differential equation by another. True, but all hope is 
not lost. 

A strategy that often works is to look for a u that is a function of x alone, or a function 
of y alone. If u is a function of x alone, that is u(x), then we write u’(x) instead of u,, and 
Uy is just zero. Then 

M, — Nz F 

=p he 
In particular, My—Ne 
a linear equation 


ought to be a function of x alone (not depend on y). If so, then we have 


, M,-Nz 


NV u= 0. 


u 
Letting p(x) = Myo Ns we solve using the standard integrating factor method, to find 
u(x) = Ce/?) 4". The constant in the solution is not relevant, we need any nonzero solution, 
so we take C = 1. Then u(x) = e/ ?()% is the integrating factor. 

Similarly we could try a function of the form u(y). Then 


M, — Nz , 
age 
In particular, My Ne ought to be a function of y alone. If so, then we have a linear equation 
ies M, —- Nz 0 
u +—+——u = 0. 
M 


Letting q(y) = My Ne we find u(y) = Ce~ JI), We take C = 1. So u(y) = e~ J is 


the integrating factor. 


Example 1.9.6: Solve 


x? + y? dy 
+ 2 = 0. 
x+1 Vode 
Solution: Let M = ze and N = 2y. Compute 
2 2 
i a ye 
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As this is not zero, the equation is not exact. We notice 


M, —N, 2y 1 1 
(2) N plop -gs1 


is a function of x alone. We compute the integrating factor 


es P(e) de = elnle+1| = a + 1]. 


Assuming that we want to look at x > —1, we multiply our given equation by (x + 1) to 


obtain 
dy 


dz 
which is an exact equation that we solved in Example 1.9.5. The solution was 


e+ y’ + 2y(x+1) 0, 


y= +, (CE. 
ob 1 
If, instead, we had wanted a solution with x < —1, we would have needed to multiply by 
—(x +1), which would have given a very similar result. | 


Example 1.9.7: Solve 


d 
y +(ayt+1)— =0. 


oy 
dx 
Solution: First compute 

M, — Nz = 2y—y =y. 


As this is not zero, the equation is not exact. We observe 


_M,-Nz _ y 1 


M yr sy 


Q(y) 


is a function of y alone. We compute the integrating factor 


e J Qu) dy — e—Iny — a 
ov] 
Therefore we look at the exact equation 


sytldy 4 
y de 


y+ 


The reader should double check that this equation is exact. We follow the procedure for 
exact equations 


F(z,y) = ary + Aly), 


and it i 
LY 1 
=2r- =r+A(y). 1.8 
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Consequently A’(y) = 7 or A(y) =Iny. Thus F(z, y) = ry + Iny. It is not possible to solve 
F (x,y) =C for y in terms of elementary functions, so let us be content with the implicit 
solution: 


syt+Iny=C. 


We are looking for the general solution and we divided by y above. We should check what 
happens when y = 0, as the equation itself makes perfect sense in that case. We plug in 
y = 0 to find the equation is satisfied. So y(x) = 0 is also a solution. | 


1.9.3. Exercises 


Exercise 1.9.2: Solve the following exact equations, implicit general solutions will suffice: 


a) (Qay +27) dx + (27+ y? +1) dy =0 b) 2 +y°@ = 


c) et +y> + 3a = d) (a + y) cos(x) + sin(x) + sin(x)y’ = 0 
Exercise 1.9.3:* Solve the following exact equations, implicit general solutions will suffice: 

a) cos(x) + ye™Y + rey’ = 0 b) (22 + y) dx + (x — 4y) dy = 0 

c) eT+eHH%=0 d) (3x? + 3y) dx + (3y? + 3x) dy =0 
Exercise 1.9.4: Solve the differential equation (2ye?*¥ — 2x) + (2re?*¥ + cos(y))y’ = 0 
Exercise 1.9.5: Solve the differential equation (—y sin(xy) —2ve” )+(—2 sin(xy) +1)y’ = 0 


Exercise 1.9.6: Solve the differential equation (2x + 3ysin(xy)) + (32 sin(xy) — e”)y’ = 0 
with y(2) = 0. 


Exercise 1.9.7: Solve the differential equation x + yy’ = 0 with y(0) = 8. Write this as an 
explicit function and determine the interval of x values where the solution is valid. 


Exercise 1.9.8: Solve the differential equation 2x — 2+ (8y+16)y’ = 0 with y(2) = 0. Write 
this as an explicit function and determine the interval of x values where the solution is valid. 


Exercise 1.9.9: Find the integrating factor for the following equations making them into 
exact equations: 


a) edz + te%dy =0 b) i dz + 3a dy = 0 


c) 4(y? + x) dx 4 2u+2u" dy = 0 d) 2sin(y) dz + x cos(y) dy = 0 
y 


Exercise 1.9.10:* Find the integrating factor for the following equations making them into 
exact equations: 


a) , dx + 3y dy =0 b) dx — e~*-¥ dy = 0 


c) (oe 1) dx 4 +, dy =0 d) (2y FY da (2y | clay =0 


y? y x 
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Exercise 1.9.11: Suppose you have an equation of the form: f(x) + gy) =). 
a) Show it is exact. 
b) Find the form of the potential function in terms of f and g. 
Exercise 1.9.12: Suppose that we have the equation f(x) dx — dy = 0. 
a) Is this equation exact? 
b) Find the general solution using a definite integral. 


Exercise 1.9.13: Find the potential function F(x, y) of the exact equation “** dx + (1/y + 
x) dy = 0 in two different ways. 


a) Integrate M in terms of x and then differentiate in y and set to N. 
b) Integrate N in terms of y and then differentiate in x and set to M. 
Exercise 1.9.14: A function u(x, y) is said to be a harmonic function if tg, + Uyy = 0. 


a) Show if u is harmonic, —u,dx + u,dy = 0 is an exact equation. So there exists (at 
least locally) the so-called harmonic conjugate function v(x, y) such that v, = —u, and 
ty = Ux: 


Verify that the following u are harmonic and find the corresponding harmonic conjugates v: 
b) uw = Qay c) u=e* cosy d) u= 2° — 3zy? 
Exercise 1.9.15:* 


a) Show that every separable equation y’ = f(x)g(y) can be written as an exact equation, 
and verify that it is indeed exact. 


b) Using this rewrite y' = xy as an exact equation, solve it and verify that the solution is 
the same as it was in Example 1.3.1. 
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1.10 Modeling with First Order Equations 


After this section, you will be able to: 


e Write a first-order differential equation to model a physical situation and 


e Interpret the solution to a differential equation in the context of a physical 
problem. 


One of the main reasons to study and learn about differential equations, particularly for 
scientists and engineers, is their application and use in mathematical modeling. Since the 
derivative of a function represents the rate of change of that quantity, if we can use physical 
or scientific principles to develop an equation for the rate of change of some quantity in terms 
of the quantity and time, there’s a chance that we can write a differential equation for this 
quantity and solve it to determine how the quantity will change. 


1.10.1 Principles of Mathematical Modeling 


The process of mathematical modeling involves three main steps. The first of these is to 
write the model. This part comes from basic science or engineering principles and involves 
writing a differential equation that fits the given situation. If we can determine the rate at 
which a quantity will change based on the surrounding factors, we have a good shot of getting 
to such an equation. One main principle that can be used to write these equations is the 
accumuilation equation, which will be discussed in the next subsection. 

The second step of this process is to solve the differential equation. This can mean either 
an analytic solution or a numeric one, and this is where the work of this class comes into 
play. We are going through a bunch of different techniques for solving differential equations 
and analyzing the overall behavior of such equations so that we can use them in this way. 
The end goal is to get an equation or a graph for how the quantity that we made a model for 
is going to change in time. 

The final step of the process is to validate the model by comparing with experimental 
data. Once we have written the model and solved the corresponding differential equation, 
we want to make sure that the model works. To do this, we can take a new version of the 
original scenario, run the model as well as the physical experiment and see how the results 
compare. If the results are “close” (in whatever sense makes logical sense for the problem), 
then we have a good model and can keep it. However, if our results differ significantly, then 
the model we used probably doesn’t apply to this problem. We need to go back to step 1 
to try to figure out a better model for the physical situation in order to get more accurate 
results. 

Why do we care about mathematical modeling? The biggest thing that it does from an 
engineering point of view is reduce the need for repeated testing. If we have a mathematical 
model that works for a given physical system, we can see how the system will be have under 
slightly different conditions and with different initial conditions without needing to run the 
physical experiment over and over again. We can do all of this testing on the model, and 
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since we have validated the model, we can assume that the actual results will be similar. This 
also allows us to change some aspects of the physical situation to try to optimize it, but do 
so just by modifying the mathematical model, not the physical setup. This can significantly 
cut down on costs and allow for more optimal system design at the same time. 


1.10.2 The Accumulation Equation 


The accumulation equation is one of the simplest general mathematical formulations that 
can be used to develop mathematical models. This equation comes down to the fact that the 
rate of change of some quantity should be equal to the rate at which it is being added minus 
the rate at which it is being removed. If we let x be the quantity in question, this can be 
written as 


= = rate in — rate out. (1.9) 
dt 

This may seem fairly simple. However, it shows up in many places in science and engineering. 

Any mass or energy balance equations are examples of accumulation equations. These types 

of equations can also be written for the accumulation of momentum, and doing so for fluids 

gives rise to the Navier-Stokes equations, providing the basis for several fields of engineering. 


The examples that we see here will be simpler than that, but the idea is still the same. 


Example 1.10.1: A tank initially contains 70 gallons of water and 5 Ibs of salt. A solution 
with salt concentration 0.2 lbs per gallon flows into the tank at a rate of 3 gal/min. The tank 
is well stirred, and water is removed from the tank at a rate of 3 gal/min. Find the amount 
of salt in the tank at any time t? What happens as t + oo? Does this make sense? 


Solution: To solve this problem, we use the accumulation equation (1.9) on the amount of 
salt in the tank. In order to compute with this, we recognize that in terms of mass of salt 
moving into the tank 

rate in = flow in x concentration in 


and similarly for the mass of salt leaving the tank. 

If we let x represent the amount of salt in the tank at any time t (which is the goal of the 
problem), we can write a differential equation for this using the accumulation equation (1.9). 
This gives us that 

dx or 

— = rate in — rate out = flow in x concentration in — flow out x concentration out 


dt 
For this problem, we have that 


flow in = 8, 
concentration in = 0.2, 


flow out = 3, 


. x x 
concentration out = = =, 
volume 70 


The last of these lines comes from the fact that the tank is “well stirred” or “well-mixed.” 
This implies that the concentration of salt in the water leaving the tank is the same as the 
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concentration in the tank, which we can compute as =". In this case, since the flow rate 


in and out are both 3 gal/min, the volume of water in the tank is fixed at 70 gallons, so we 
can put this in the equation. 
Therefore, our equation becomes 


dx x 
~ = (3x 0.2) — (3 =). 
rae a eal ad 7 
We can rewrite this equation as 
i Daly 
—+—2z=0. 
dt 70 


which we recognize as a first order linear equation. We can then solve this using the method 
of integrating factors. Our factor r(t) is 


r(t) — ed P(t) at — el 70 tt — ert 


which we can multiply on both sides of the equation to obtain 


The left side of this is a product rule derivative, so we can integrate both sides to obtain 
erg = 0.6 ern! +C. 
We can then isolate x to get our general solution as 
w= 144+ Cer", 
Our initial condition tells us that x(0) = 5. Plugging this in gives that 
5=2(0)=144+C => C=-%9, 


so the solution to the initial value problem, and thus our calculation for the amount of salt 
in the tank at any time ¢, is 


a(t) = 14 — 9e~70". 


As t + oo, we see that the exponential term goes to zero. This leaves us with 14 lbs 
of salt in the tank after a long time. This makes some sense because this would give us 
a concentration of a = 0.2 lb/gal, and that was exactly the concentration of the in-flow 
stream. It makes sense that after a long time of mixing and removing water from the tank, 
the concentration of the tank would match that of the incoming stream. _| 


The same principle works for other types of examples, including those where the volume 
of the tank is not constant in time. 
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Example 1.10.2: A 100 liter tank contains 10 kilograms of salt =) 5 L/min, 0.1 ke/L 
dissolved in 60 liters of water. Solution of water and salt (brine) with » 
concentration of 0.1 kilograms per liter is flowing in at the rate of 
5 liters a minute. The solution in the tank is well stirred and flows 
out at a rate of 3 liters a minute. How much salt isin the tank when = \-_A_A_J 


the tank is full? 60 L 

Solution: We can again use the accumulation equation to write 10 kg salt = 
\\ 

dx 

aE (flow in x concentration in) — (flow out x concentration out). 3 U/min 


In this example, we have 


flow in = 5, 
concentration in = 0.1, 


flow out = 3, 
Ms 


ci 
volume 60+ (5—3)t' 


concentration out = 


Our equation is, therefore, 


Or in the form (1.3) 
dx 3 


a+ 
di ' 60+2t" 
Let us solve. The integrating factor is 


= 3 _ 3 7 3/2 
r(t) = exp (| SOx st) exp (; In(60 + 2) = (604 a0)". 


We multiply both sides of the equation to get 


dx 3 
602-97)" — 4 609 — 0.5(60 + 2t)*/? 
aaa ae t | +2) 60 + 2t” ae 
d 
; (60 a. 24°72 = 0.5(60 + 2¢)°/?, 


(60 + 2t)*/?2 = [.5(00 re ales aoe Os 


5 | (60 + 2t)*”? 
2 


x = (60 + 2t) ae Cli .on 


1 
x = (60+ at) P= (60 455 SO COI. 


60+ 2 
 —— 
10 


+ C(60 + 2t)-*”. 
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We need to find C. We know that at 
f=0, = 10,50 


60 
10 = x(0) = 7p tO (60) = 6+C(60)*?, 
or 


C = 4(60°/*) = 1859.03. 


We are interested in x when the tank is 
full. The tank is full when 60 + 2¢ = 100, or 
when t = 20. So ® 8 ‘0 8 ~ 


Figure 1.81: Graph of the solution x kilograms 
60 + 40 _ J P 
i0 + C(60 + 40) ne of salt in the tank at time t. 


~ 10 + 1859.03(100)~*”” & 11.86. 


2(20) = 


See Figure 1.31 for the graph of x over t. 
The concentration when the tank is full is approximately 0.1186 */liter, and we started 
with 1/6 or 0.167 ke/iter. | 


For the previous example, we obtained the solution 


_ 60+2t 


a(t) + 1859.03(60 + 2t)~3/?, 


which is valid and well defined for all positive values of t¢ (it has an issue at t = —30, but 
we aren’t concerned about that here). However, as a differential equation that represents a 
physical situation, it is not valid for all positive values of t. The issue here is that the tank is 
full at t = 20. Therefore, beyond this point, while the function still exists, it is not a valid 
model for this physical system. Once the tank fills, you can’t keep adding and removing 
water at the same rates that you have been up until this point, because something is going 
to break with the system. The same goes for if you are removing water from the tank at a 
faster rate than you are adding it, because then the tank will empty at some point in time 
and beyond that, the model equation no longer represents the system. 

The same ideas apply to problems involving interest compounded continuously. For an 
interest rate of r, the “rate in,” or the rate at which the money in the account is increasing, 
is rP where P is the amount of money in the account. Taking this along with other factors 
that may affect the balance of the account allows us to write a differential equation, which 
we can solve to determine what the balance will be over time. 


Example 1.10.3: A bank account with an interest rate of 6% per year, compounded con- 
tinuously, starts with a balance of $30000. The owner of the account withdraws $50 from the 
account each month. Find and solve a differential equation for the account balance over time. 
What is the largest amount that the owner could withdraw each month without the account 
eventually reaching $0? 


Solution: We will use the function P(t) to model the balance of the account over time, 
where ¢ is in years. Since the owner withdraws $50 per month, this means that they withdraw 
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$600 over the course of the year. This means that the differential equation we want is 


dP 
oe 0.06P — 600 P(0) = 30000. 


We can solve this equation by the integrating factor method. 


P’ — 0.06P = —600 
(eo Oreepy! Ss —600e~ 2-0 
eu = 10000e ee 
P = 10000 + Ce?™ 


For P(0) = 30000, we need to take C = 20000. Thus, the solution to the initial value problem 
is 
P(t) = 10000 + 20000c°. 


0.06t is positive, this means that the account balance here 


Since the coefficient in front of e 
will grow in time. 

For the second part, we need to adjust the withdrawal amount to see how the solution 
changes. If we let K be the monthly withdrawal amount, then we have the differential 


equation 


dP 
aie 0.06P — 12K P(0) = 30000. 
The same solution method gives us 
12K 
P = 0.06¢ | 
(t) aOR + Ce 


If C < 0, then the account balance will eventually go to zero. Therefore, we need C' > 0, 
and since P(0) = 30000, we have that 


12k 1k 
30000 = ——+C or C= 30000 —- —. 
0.06 0.06 


For this to be equal to zero, we need 


12k 
—— = 30000 k = 150. 
0.06 


Thus, the owner can withdraw $150 per month and keep the account balance positive. __| 


To end this section, we will analyze the example that was presented at the very beginning 
of the book. 


Example 1.10.4: An object falling through the air has its velocity affected by two factors: 
gravity and a drag force. The velocity downward is increased at a rate of 9.8 m/s? due to 
gravity, and it is decreased by a rate equation to 0.3 times the current velocity of the object. 
If the ball is initially thrown downwards at a speed of 2 m/s, what will the velocity be 10 
seconds later? 
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Solution: As described in that first section, we know that the differential equation that we 
can write for this situation is 


and that the initial condition for the velocity if v(0) = 2. Since we have gravity as a positive 
9.8, this means that the downward direction is positive, so the object being thrown downward 
at 2 m/s means that it is positive. We then need to solve this initial value problem, which 
we can do using first order linear methods. The equation can be written as 


v + 0.3v = 9.8 

which has integrating factor e?**. After multiplying this to both sides and integrating, we 
get that 

9.8 

0.3t 0.3¢ 
ey =—e"4+C 

0.3 

or that 


9.8 
v(t) = a Ge. 


Using the initial condition, we get that 
98 


v(0) =—+C=2 
3 
so that C = —2 and the solution to the initial value problem is 
_ 98 92 0.38 


t a 
ol er aearlere 
Then, to determine the velocity at t = 10, we can plug 10 into this formula to get that 


v(10) = — — as x 31.14 m/s. 
zl 


All of these examples are based around the same idea of the accumulation equation. We 
need to determine the quantity that is changing as well as all of the factors that cause it to 
increase and decrease. These get combined into a differential equation which we can solve in 
order to analyze the situation and answer whatever questions you want about that physical 
problem. Keeping these ideas in mind will help you approach a wide variety of problems 
both in this class as well as future applications in engineering classes and beyond. 


1.10.3. Exercises 


Exercise 1.10.1: Suppose there are two lakes located on a stream. Clean water flows into 
the first lake, then the water from the first lake flows into the second lake, and then water 
from the second lake flows further downstream. The in and out flow from each lake is 500 
liters per hour. The first lake contains 100 thousand liters of water and the second lake 
contains 200 thousand liters of water. A truck with 500 kg of toxic substance crashes into the 
first lake. Assume that the water is being continually mixed perfectly by the stream. 
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a) Find the concentration of toxic substance as a function of time in both lakes. 
b) When will the concentration in the first lake be below 0.001 kg per liter? 
c) When will the concentration in the second lake be maximal? 


Exercise 1.10.2: Newton’s law of cooling states that 4 = —k(x — A) where x is the 
temperature, t is time, A is the ambient temperature, and k > 0 is a constant. Suppose that 
A = Ap cos(wt) for some constants Aj and w. That is, the ambient temperature oscillates 
(for example night and day temperatures). 


a) Find the general solution. 


b) In the long term, will the initial conditions make much of a difference? Why or why 
not? 


Exercise 1.10.3: Initially 5 grams of salt are dissolved in 20 liters of water. Brine with 
concentration of salt 2 grams of salt per liter is added at a rate of 3 liters per minute. The 
tank is mixed well and is drained at 3 liters per minute. How long does the process have to 
continue until there are 20 grams of salt in the tank? 


Exercise 1.10.4: Initially a tank contains 10 liters of pure water. Brine of unknown (but 
constant) concentration of salt is flowing in at 1 liter per minute. The water is mixed well 
and drained at 1 liter per minute. In 20 minutes there are 15 grams of salt in the tank. What 
is the concentration of salt in the incoming brine? 


Exercise 1.10.5:* Suppose a water tank is being pumped out at 34/min. The water tank 
starts at 10L of clean water. Water with toxic substance is flowing into the tank at 2¥/nin, 
with concentration 20t8/L at time t. When the tank is half empty, how many grams of toxic 
substance are in the tank (assuming perfect mixing)? 


Exercise 1.10.6: A 300 gallon well-mixed water tank initially starts with 200 gallons of 
water and 15 Ibs of salt. One stream with salt concentration one pound per gallon flows into 
the tank at a rate of 3 gallons per minute and water is removed from the well-mixed tank at 
a rate of 2 gallons per minute. 


a) Write and solve an initial value problem for the volume of water in the tank at any 
time t. 


b) Set up an initial value problem for the amount of salt in the tank at any time t. You 
do not need to solve it (yet), but should make sure to state it fully. 


c) Is the solution to this initial value problem a valid representation of the physical model 
for all times t > 0? If so, use the information in the equation to determine the long-time 
behavior of the solution. If not, explain why, determine the time when the representation 
breaks down, and what happens at that point in time. 


d) Solve the initial value problem above and compare this to your answer to the previous 
part. 
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Exercise 1.10.7: A 500 gallon well-mixed water tank initially starts with 300 gallons of 
water and 200 Ibs of salt. One stream with salt concentration of 0.5!/gal flows into the tank 
at a rate of 5&4l/min and water is removed from the well-mixed tank at a rate of 74!/min. 


a) Write and solve an initial value problem for the volume of water in the tank at any 
time t. 


b) Set up an initial value problem for the amount of salt in the tank at any time t. You 
do not need to solve it (yet), but should make sure to state it fully. 


c) Is the solution to this initial value problem a valid representation of the physical model 
for all times t > 0? If so, use the information in the equation to determine the long-time 
behavior of the solution. If not, explain why, determine the time when the representation 
breaks down, and what happens at that point in time. 


d) Solve the initial value problem above and compare this to your answer to the previous 
part. 


Exercise 1.10.8: A 200 gallon well-mixed water tank initially starts with 150 gallons of 
water and 50 Ibs of salt. One stream with salt concentration of 0.21>/gai flows into the tank at 
a rate of 48@!/min and water is removed from the well-mixed tank at a rate of 4&4!/min. 


a) Write and solve an initial value problem for the volume of water in the tank at any 
time t. 


b) Set up an initial value problem for the amount of salt in the tank at any time t. You 
do not need to solve it (yet), but should make sure to state it fully. 


c) Is the solution to this initial value problem a valid representation of the physical model 
for all times t > 0? If so, use the information in the equation to determine the long-time 
behavior of the solution. If not, explain why, determine the time when the representation 
breaks down, and what happens at that point in time. 


d) Solve the initial value problem above and compare this to your answer to the previous 
part. 


Exercise 1.10.9:* Suppose we have bacteria on a plate and suppose that we are slowly 
adding a toxic substance such that the rate of growth is slowing down. That is, suppose that 
i = (2—0.1t)P. If P(0) = 1000, find the population at t = 5. 


Exercise 1.10.10:* A cylindrical water tank has water flowing in at I cubic meters per 
second. Let A be the area of the cross section of the tank in meters. Suppose water is flowing 
from the bottom of the tank at a rate proportional to the height of the water level. Set up 
the differential equation for h, the height of the water, introducing and naming constants 
that you need. You should also give the units for your constants. 


Exercise 1.10.11: An object in free fall has a velocity that increases at a rate of 32 ft/s?. 
Due to drag, the velocity decreases at a rate of 0.1 times the velocity of the object squared, 
when written in feet per second. 
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a) Write a differential equation to model the velocity of this object over time. 


b) This equation is autonomous, so draw a phase diagram for this equation and classify 
all critical points. 


c) What will happen to the velocity if the object is dropped at t = 0? What about if the 
object is thrown downwards at a rate of 10 ft/s? 


Exercise 1.10.12: The number of people in a town that support a given measure decays at 
a constant rate of 10 people per day. However, the support for the measure can be increased 
by individuals discussing the issue. This results in an increase of the support at a rate of 
ay(1000 — y) people per day, where y is the number of people who support the measure, 
and a is a constant depending on the way in which the issue is being discussed. Write a 
differential equation to model this situation, and determine the amount of people who will 
support the measure long-term if a is set to 2. 


Exercise 1.10.13: Newton’s Law of Procrastination states that the rate at which one 
accomplishes a chore is proportional to the amount of the chore not yet done. Unbeknownst 
to Newton, this applies to robots too. A Roomba is attempting to vacuum a house measuring 
1000 square feet. When none of the house is clean, the roomba can clean 200 square feet per 
hour. What makes this problem fun is that there is also a dog. It’s whatever kind of dog you 
like, take your pick. The dog dirties the house at a constant rate of 50 square feet per hour. 


a) Assume that none of the house is clean at t = 0. Write a DE for the number of square 
feet that are clean as a function of time, and solve for that quantity. 


b) How long will it take before the house is half clean? Will it ever be entirely clean? 
(Explain briefly.) 


Exercise 1.10.14: A student has a loan for $50000 with 5% interest. The student makes 
$300 payments on the loan each month. 


a) With this setup, how long does it take the student to pay off the loan? How much 
money does the student pay over this period of time? 


b) What is the minimal amount the student should pay each month if they want to pay 
off the loan within 5 years? How much does the student pay over this period? 


Exercise 1.10.15: A factory pumps 6 tons of sludge per day into a nearby pond. The pond 
initially contains 100,000 gallons of water, and no sludge. Each day, 3,000 gallons of rain 
water falls into the pond, and 1,000 gallons per day leave the pond via a river. Assume, for 
no good reason, that the water leaving the pond has the same concentration of sludge as the 
pond as a whole. How much sludge will there be in the pond after 150 days? 


Exercise 1.10.16: In this exercise, we compare two different young people and their 
investment strategies. Both of these people are investing in an account with 7.5% annual 
rate of return. Person 1 invests $50 a month starting at age 20, and Person 2 invests $100 
per month starting at age 30. Write differential equations to model each of these account 
balances over time, and compute the amount of money in each account at age 50. Who has 
more money in the account? Who has invested more money? What would person 2 have to 
invest each month in order for the two balances to be equal at age 50? 
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Exercise 1.10.17: Radioactive decay follows similar rules to interest, where a certain portion 
of the material decays over time, resulting in an equation of the form 


dy - 
dt 
for some constant k. The half-life of a material is the amount of time that it takes for half 


of the material to have decayed away. Assume that the half-life of a given substance is T 
minutes. Find a formula for k, the coefficient in the decay equation, in terms of T. 


—ky 
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1.11 Modeling and Parameter Estimation 


After this section, you will be able to: 


e Use parameter estimation to approximate physical parameters from data. 


One of the most common ways that the mathematical modeling structure can be used 
to analyze physical problems is the idea of parameter estimation. The situation is that we 
have physical principles that give rise to a differential equation that defines how a physical 
system should behave, but there are one or more constants in the problem that we do not 
know. Two simpler examples of this are Newton’s Law of Cooling 


dT 

dt 
which models the temperature of an object in an environment of temperature 7’, over time, 
and velocity affected by drag 


=k(P a 7) 


= 9.8 — av’ 

modeling the velocity of a falling object where the drag force is proportional to the square of 
the velocity. In both of these cases, the models are well established, but for a given object, 
we likely do not know the & or a values in the problem. These are these parameters of the 
problem, and would be determined by the shape and structure of the objects, the material 
that it is made of, and many other factors, so it could be hard to figure out what they are in 
advance. How can we find these values? We can use data from the actual physical problem 
to try to estimate these parameters. 

The easier version of this is to use a single value at a later time to calculate the constant. 


Example 1.11.1: An object that obeys Newton’s Law of Cooling is placed in an environment 
at a constant temperature of 20° C. The object starts at 50° C, and after 10 minutes, it has 
reached a temperature of 40° C. Find a function for the temperature as a function of time. 


Solution: Based on Newton’s Law of Cooling, we know that the temperature satisfies the 
differential equation 

dT 

dt 
with initial condition T(0) = 50, but we do not know the value of k. In order to work this 
out, we should solve the differential equation with unknown constant k, then figure out which 
value of k gives us the appropriate temperature after 10 minutes. This is a first order linear 
equation, which can be rewritten as 


“rear 320) 


T' +kT = 20k. 
The integrating factor we need is e*, which turns the equation into 


(eT) = 20ke™. 
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Integrating both sides and solving for T’ gives 
T(t) = 20+ Ce, 


To satisfy the initial condition, we need that T(0) = 50, or C = 30. Thus, our solution, 
still with an unknown constant k, is 


T(t) = 20 + 30e-™. 


To determine the value of k, we need to utilize the other given piece of information: that 
T(10) = 40. Plugging this in gives that 


40 = 20 + 30e7 1% 


which we can solve for k using logarithms. This will give that 


2 1 2 
ese C0 — k = ——l1n-. 
5 10, 3 
Finally, we can plug that constant into our equation to get the solution for the temperature 
at any time value, : 
T(t) = 20 + 30e7 105, 
0 4 
This method works great if we have the exact measurement from the object at one point 
in time. However, if the measurements at multiple points in time are known, and if the data 
is not likely to be exact, then a different method is more applicable. The idea is that we want 
to minimize the “error” between our predicted result and the physical data that we gather. 
The method used to minimize the error is the “Least Squared Error” method. 
Assume that we want to do this for the drag coefficient problem, 
du 
— =9.8-— av’ 
dt 
where we do not know, and want to estimate, the value of a. For this method, the data 
that we gather is a set of velocity values vj, vo,...,Un, that are obtained at times f1, to, ..., tn. 
For any given value of a, we can solve, either numerically or analytically, the solution vg 
to the given differential equation with that value of a. From this solution, we can compute 
Va(t1), Va(t2), ..-; Va(tn), the value of this solution at each of the times that we gathered data 
originally. Now, we want to compute the error that we made in choosing this parameter a. 
This is computed by 


F(a) = (v1 = te(t1))? + (v2 = valte))? +++ (un, — valtr))? 


which is the sum of the squares of the differences between the gathered data and the predicted 
solution. In order to find the best possible value of a, we want to minimze this error by 
choosing different values of a 


Emin = min E(a) = min Sov — Va(t;))” 
i=l 
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and whatever value of a gives us this minimum is the optimal choice for that parameter. 


The function that we want to minimize here is usually a very complicated function, and 
we may not even be able to solve the differential equation analytically for any a. Thus, 
computers are used most often here to solve these types of problems. 


Example 1.11.2: An object is falling under the force of gravity, and has a drag component 
that is proportional to the square of the velocity. Data is gathered on the falling object, and 
the velocity at a variety of times are given in Table 1.3. 


t (s) | v (m/s) 
0 0 
0.1 | 0.9797 
0.3 | 2.8625 
0.5 | 4.4750 
0.8 | 6.3828 
0.9 | 6.8360 
1.0 | 7.0334 
1.5 | 8.1612 


Table 1.8: Data for estimating drag coefficient using least squared errors. 


Use this data to estimate the coefficient of proportionality on the drag term in the equation 


d 
= =98—av’. 


Solution: To solve this problem, we will use the least squared error method implemented in 
MATLAB. The code we need for this is the following, which makes use of the Optimization 
Toolbox. 


global tVals 
global vVals 


BWA = (0; Oi, O.8, O-5, O.8, O.9, 1.0, 1.8% 
vVals [0,0.9797,2.8625,4.4750,6.3828,6.8360,7.0334,8.1612]; 


[aVal, errVal] = fminbnd(@(a) EstSqError(a), 0, 4) 


This bit of code inputs the necessary values and uses the fminbnd function to find the 
minimum of the error function on a defined interval. These problems need to be done on a 
bounded interval, but in most physical situations there is some reasonable window for where 
the parameter could be. The rest of the code is the definition of the EstSqError function. 
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function err = EstSqError(al) 


global tVals 
global vVals 


fun = @(t,v) 9.8 - al.*v.72; 
sol = ode45(fun, [0,3], 0); 
vTest = deval(sol, tVals); 


err = sum((vVals - vTest).~2) 
end 


The main point of this code is that it takes in a value of a, over which we are trying to 
minimize, it numerically solves the differential equation with that value of @ over a desired 
range of values, and then compares the inputted vVals with the generated vTest array, 
computing the sum of squared errors, and returning the error value. 

Running this code results in an a value of 0.1256, with an error of 0.0345. That means 
that, based on this data, the best approximation to a is 0.1256. _| 


Note that in the above example, the total error was not zero, and doesn’t actually match 
the coefficient used to generate the data, which was 0.123. This is because noise was added 
to the data before trying to compute the drag coefficient. In a real world problem, noise 
would not be added, but a similar effect would arise from slightly inaccurate measurements or 
round-off errors in the data. While we may not have found the constant exactly, we got really 
close to it, and could use this as a starting point for further experiments and data validation. 


1.11.1 Exercises 


Exercise 1.11.1: Bob is getting coffee from a restaurant and knows that the temperature 
of the coffee will follow Newton’s Law of Cooling, which says that 


dT’ 


where To is the ambient temperature and k is a constant depending on the object and 
geometry. His car is held at a constant 20° C, and when he receives the coffee, he measures 


the temperature to be 90° C. Two minutes later, the temperature is 81°C. 


a) Use these two points of data to determine the value of k for this coffee. 


b) Bob only wants to drink his coffee once it reaches 65° C. How long does he have to wait 
for this to happen? 


c) If the coffee is too cold for Bob’s taste once it reaches 35° C, how long is the acceptable 
window for Bob to drink his coffee? 
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Exercise 1.11.2: Assume that a falling object has a velocity (in meters per second) that 


obeys the differential equation 


d 
= 98-0 


where a represents the drag coefficient of the object. 


a) Solve this differential equation with initial condition v(0) = 0 to get a solution that 
depends on a. 


b) Assume that you drop an object from a height of 10 meters and it hits the ground after 
3 seconds. What is the value of a here? (Note: You solved for v(t) in the previous part, 
and now you need to get to position. What does that require?) 


c) Assume that a second object hits the ground in 6 seconds. How does this change the 
value of a? 


Exercise 1.11.3: A restaurant is trying to analyze the to-go coffee cups that it uses in order 
to best serve their customers. They assume that the coffee follows Newton’s Law of Cooling 
and place it in a room with ambient temperature 22° C. They record the following data for 
the temperature of the coffee as a function of time. 


t (min) | T (° C) 
0 80 
0.5 77.1624 
1.1 73.8082 
Lt 70.6800 
2.3 67.7996 


a) Use code to determine the best-fit value of k for this data. 


b) The restaurant determines that to avoid any potential legal issues, the coffee can be no 
warmer than 60 °C when it is served. If the coffee comes out of the machine at 90 °C, 
how long do they have to wait before they can serve the coffee? 
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Exercise 1.11.4:* Assume that an object falling has a velocity that follows the differential 


equation 


d 
a =98—av" 


where the velocity is in ™/s and a represents the drag coefficient. During a physics experiment, 
a student measures data for the velocity of a falling object over time given in the table below. 
Use this data (and code) to estimate the value of a for this object. 


t (s) | v (m/s) 
0 0 

a4). gees t P ae 
0.2 1.9341 7 58.6556 
0.4 3.6597 

14 68.4521 
0.6 5.1613 

28 91.4883 
0.9 6.7847 

on 108.5750 
ed 7.4103 

50 135.7148 
1.3 7.9471 

78 197.3520 
Be | eee 100 | 239.9479 
1.8 8.5739 
2.1 | 8.7769 Table 1.5: Data for Exercise 1.11.5. 


Table 1.4: Data for Exercise 1.11.4. 


Exercise 1.11.5: Assume that a species of fish in a lake has a population that is modeled 


by the differential equation 

dP 1 

— = ——rP(K — P)- 

dt ~ Too"! | as 
where r, K, and @ are parameters, r representing the growth rate, K the carrying capacity, 
and a the harvesting rate, and the population P is in thousands., with t given in years. From 
previous studies, you know that the best value of r is 3.12. After studying the population 


over a period of time, you get the data given above. 


a) Your friend tells you that in a previous study, he found that the value of K for this 
particular lake is 255.2. Use code to determine the best value of a for this situation. 
Note that the equation expects t in years, but the data is given in days. 


b) That answer doesn’t look great. Plot the solution with these parameters along with 
the data and compare them. 


c) The fit does not look great, so maybe your friend’s value was not quite right. Run code 
to find best values for K and a simultaneously. 
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1.12 Substitution 


Attribution: [JL], §1.6. 


After this section, you will be able to: 


e Use substitution to solve more complicated first order equations, 
e Use a Bernoulli substitution to solve appropriate first order equations, and 


e Use a homogeneity transformation to solve appropriate first order equations. 


The equation 
yf = («-y +1) 
is neither separable nor linear. What can we do? One technique that worked for helping 
us in evaluating integrals was substitution, or change of variables. For example, in order to 
evaluate the integral 


pre + 4)° dx 


we can do so by defining u = x7 + 4 so that du = 2x dx, and then evaluate the integral as 


6 2 6 
feu=Fec-F 49 40 


after we have plugged our original function back in. 

We can try to do the same thing here, and be careful with how we set things up. Our 
general strategy will be to pick a new dependent variable, find a differential equation that 
this new variable solves (which will come from the old equation), solve that equation, then 
convert back to the original variable. We will take v as our new dependent variable here, 
which is as function v(x). Let us try 


v=2—ytl, 


which is the “inside” function (it’s inside the square) of our example. In order to get to a 
differential equation that v satisfies, we need to figure out y’ in terms of v’, v and x. We 
differentiate (in x) to obtain v' = 1—y'. So y’ =1—v'. We plug this into the equation to get 


l—-v' =v". 


In other words, v’ = 1 — v?. Such an equation we know how to solve by separating variables: 


1 
[og = ee. 
re 1 1 1 1 
i vt eee! as utl = e2et2e an u+ = De, 
2, v—1 v—1 v—1 
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for some constant D. Note that v = 1 and v = —1 are also solutions; they are the singular 
solutions in this problem. (This solution method requires partial fractions; for a review of 
that topic, see § B.3.) 

Now we need to “unsubstitute” to obtain 


and also the two solutions x —-y+1=1lory=2,andz—y+1=-lory=2z+2. We 
solve the first equation for y. 


z—y+2= (x —y)De™, 
z—y+2= Daxe™ — yDe*, 
=y +yDe" = Dre™ — x = 2, 
y (—1+ De**) = Dre** — x — 2, 
Dae** —x—2 
De** —1 


Y= 


Note that D = 0 gives y = x + 2, but no value of D gives the solution y = «. 


Substitution in differential equations is applied in much the same way that it is applied 
in calculus. You guess. Several different substitutions might work. There are some general 
patterns to look for. We summarize a few of these in a table. 


When you see ‘Try substituting 


yy! vay 
yy! vay? 
(cos y)y! v=siny 
(sin y)y’ Uv = Cosy 
y'e¥ v=e¥ 


Usually you try to substitute in the “most complicated” part of the equation with the 
hopes of simplifying it. The table above is just a rule of thumb. You might have to modify 
your guesses. If a substitution does not work (it does not make the equation any simpler), 
try a different one. 


1.12.1 Bernoulli equations 


There are some forms of equations where there is a general rule for substitution that always 
works. One such example is the so-called Bernoulli equation*: 


y' + p(x)y = q(x)y”. 


*There are several things called Bernoulli equations, this is just one of them. The Bernoullis were a 
prominent Swiss family of mathematicians. These particular equations are named for Jacob Bernoulli 
(1654-1705). 
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This equation looks a lot like a linear equation except for the y”. If nm = 0 or n = 1, then the 
equation is linear and we can solve it. Otherwise, the substitution v = y!~” transforms the 
Bernoulli equation into a linear equation. Note that n need not be an integer. 


Example 1.12.1: Find the general solution of 


4 2 
Wome, Pia od, mae 4 
y 3x2 34 
Solution: This equation fits the Bernoulli equation structure with p(x) = —# and q(x) = —3. 


Since there is a y* on the right-hand side, we take n = 4 and make the substitution 
v=y'*=y-°. With this, we see that 


y! = —3y~4y’ 


or y’ = —1/sy*v'. Plugging this into the equation gives 


Tats: i eG 
34 a 8 
(eee 
3 aE 

<< 
v+—v=2 
x 


This last equation is now a first order linear equation, so we can solve it. The integrating 
factor we are looking for is 


p(x) = ed Pt) dx __ ale da __ cilne _ xt 
which results in the euation 
xy! + 4a3y = 22%. 
The left-hand side is (x*v)’, so we can integrate both sides to get 


2 


A es eas C 
xuvu 5 +, 
or, solving for v, 
(es) 2 iD C 
viv) = -@+—. 
5 a 


However, our original equation was for y, not v. Using the fact that v = y~°, we can solve 
for yas y= v—'/3, giving 
as 6S a 1 
x)= (| sr4+ — = ——— 


as the general solution to this equation. _| 


Even if we need to use integrals to write out the solution to these Bernoulli equations, we 
can still use the substitution method and solve back out for the desired solution at the end. 
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Example 1.12.2: Solve 
zy +y(2+1)4 ay =0, vl ==, 


Solution: First, the equation is Bernoulli (p(x) = (a + 1)/a and q(a) = —1). We substitute 


v= y? = y, y! = —Ayy’. 


In other words, (~1/4) y°v’ = y’. So 


zy +y(a +1) + ay? = 0, 
4 


Su ty (etl +e=0, 


Sul +u(z+1)+2=0, 


v' +y(x +1) +2y? =0, 


and finally 
y set), 
% 


= 4, 


The equation is now linear. We can use the integrating factor method. In particular, we use 
formula (1.4). Let us assume that x > 0 so |z| = x. This assumption is OK, as our initial 


condition is x = 1. Let us compute the integrating factor. Here p(s) from formula (1.4) is 
—4(s+1) 


x —4a+4 
efi P() 45 — exp (| =4(a 1) is) =e 4In(x)+4 _ 4a+4,—4 _ € . 
1 Ss 


x 
e~ Jy p(s)ds __ eitt4ln(2)—4 4a—4 a 


We now plug in to (1.4) 


u(x) = e Sr Pls) ds (| eli PS) 484 che + i) 


1 


eT p—4t+4 
= e**- 444 (| a= dt ++ i) . 
al 


The integral in this expression is not possible to find in closed form. As we said before, it is 
perfectly fine to have a definite integral in our solution. Now “unsubstitute” 


x e 4t+4 
y 4 — eit-4,4 (« / rm dt + i) ' 
1 


e ttl 


y= : 
a(4 f? a dt +1)" 


108 CHAPTER 1. FIRST ORDER DIFFERENTIAL EQUATIONS 


1.12.2 Homogeneous equations 


Another type of equations we can solve by substitution are the so-called homogeneous 
equations. Note that this is not the same as a homogeneous linear equation. These equations 
do not have to be linear, and are solved in a very different way. Suppose that we can write 
the differential equation as 
y =F (2) 
z 


ae and therefore y =v+au'. 
L 


Here we try the substitutions 


We note that the equation is transformed into 
1 
vtav=F(v) or av’ =F(v)-v or mw e=-. 
x 


Hence an implicit solution is 
i ! dv = In|z|+C 
——— dv =In|zx 
F(v)-—v 


ry=ytery, y(lj=1. 


Example 1.12.3: Solve 


Solution: We put the equation into the form y! = (¥/z)? + ¥/x. We substitute v = 4/2 to get 
the separable equation 
gu =vrtu-—vav’, 


which has a solution 


[qve=inte +C, 
v 


-1 
— =In|z|+C, 
v 
-1 
v= ————.. 
In |x| +C 
We unsubstitute 
Us So 
x Injal|+C’ 
= oe 
4 tn lz} + C° 
We want y(1) = 1, so 
ee eee 
ae cam eh ramen 
Thus C' = —1 and the solution we are looking for is 
i. Je 
oth lz} — 1° 
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1.12.3 


Exercises 


Hint: Answers need not always be in closed form. 


Exercise 
Exercise 
Exercise 
Exercise 
Exercise 
Exercise 
Exercise 
Exercise 
Exercise 
Exercise 
Exercise 


Exercise 


Exercise 


Exercise 


1.12.1: Solve y' + y(x? — 1) + xy® = 0, with y(1) = 1. 
1.12.2:* Solve xy’ +y+y? =0, y(1) =2. 

1.12.3: Solve 2yy’ +1 =y?+2, with y(0) =1. 
1.12.4:* Solve zy’ +y+a2=0, y(1) =1. 

1.12.5: Solve y’ + ry = y*, with y(0) = 1. 

1.12.6: Solve y! + 3y = 2ry*. 

1.12.7: Solve ry’ — 2y = (32? — 2~3)y? with y(1) = 2. 
1.12.8: Solve y’ + 5y = S. 
1.12.9:* Solve y’*y' = y* — 32, y(0) = 2. 
1.12.10: Solve yy’ +2 = fx? + y?. 
1.12.11: Solve y! = (4+y—1)’. 


1.12.12: Solve y! = ==", with y(1) = 2. 


1.12.13:* Solve 2yy! = e—* + 2x. 


1.12.14: Consider the DE 


dy ie 4 
a (u 7 *) = (1.10) 


a) Explain why (1.10) is not a linear equation. 


b) Use a Bernoulli substitution to solve (1.10). 
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Chapter 2 


Higher order linear ODEs 


As addressed in Chapter 1, we have a lot of different techniques for solving first order 
equations. However, not all differential equations are first order. A lot of physical systems 
in the world operate using higher order equations, particularly second order. Consider the 
system of a mass hanging from a spring. Newton’s second law tells us that the net force on 
the object equals the mass of the object times its acceleration. However, Hooke’s law for 
springs says that the force the spring exerts on the object is proportional to the distance this 
object is from the equilibrium position. Therefore, we get a relation between the acceleration 
of the object and the position. Since the acceleration is the second derivative (in time) of the 
position of the object, this naturally gives rise to a second order equation. 

This means that we want to see what we can do with higher order equations as well. If we 
can manage to find solutions to these equations as well, then we can address more types of 
physical problems as well. However, increasing the order of the equation makes it significantly 
more difficult to find solutions. Even for linear equations, where in first order, we had an 
explicit method and formula for solutions, we need to put many more restrictions on higher 
order linear equations in order to have a direct method to generate solutions. 


2.1 Second order linear ODEs 


Attribution: [JL], §2.1. 


After this section, you will be able to: 


Identify the general second order linear differential equation, 


Determine the characteristic equation for constant coefficient equations, 

Find the general solution for constant coefficient equations in the real and distinct 
roots case, and 

Determine if two functions are linearly independent. 
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The general second order ordinary differential equation is of the form 
y’ = F(z,y,y’) 


for F’ an arbitrary function of three variables. As with first order equations, if the function F’ 
is not in a nice or simple form, there really isn’t a hope to find a solution for this. For second 
order equations, we need to be even more specific about the structure of these equations in 
order to find solutions than we did for first order. 

Definition 2.1.1 


The general second order linear differential equation is of the form 


Aly 1 Bla)y -Cia)y = F(a): 


This equation can be written in standard form by dividing throuhg by A(z) to get 


OP a ae GI) (2.1) 


where p(x) = 3/4), q() = C@/a@), and f(x) = F@/A@). 


The word linear means that the equation contains no powers nor functions of y, y’, and 
y”. In the special case when f(x) = 0, we have a so-called homogeneous equation 


y" + p(a)y’ + a(a)y = 0. (2.2) 
We have already seen some second order linear homogeneous equations. 


y' +k’y =0 Two solutions are: y; =cos(kx), yo = sin(ka). 
y" —k’y =0 Two solutions are: y, =e", yo=e*. 

With the examples above, we were able to find solutions. However, notice that these 
equations don’t have functions of x as coefficients of the y term. This means they are constant 
coefficient equations. It turns out that one of the few ways we can have a guaranteed method 
for finding solutions to these equation is if they have constant coefficients. For first order, 
we had a method for every linear equation, but for second order, we only have a formulaic 
method for constant coefficient and homogeneous linear equations. 

If we know two solutions of a linear homogeneous equation, we know many more of them. 


Theorem 2.1.1 (Superposition) 


Suppose y; and y are two solutions of the homogeneous equation (2.2). Then 


y(x) = Ciyr(x) + Coy2(z), 


also solves (2.2) for arbitrary constants C; and C4. 


That is, we can add solutions together and multiply them by constants to obtain new 
and different solutions. We call the expression Cy, + Coy2 a linear combination of y; and yo. 
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Let us prove this theorem; the proof is very enlightening and illustrates how linear equations 
work. 


Proof: Let y = Cyy, + Coy2. Then 


y" + py’ + ay = (Cry, + Caye)” + p(Ciyr + Caya)! + a(Ciys + C2y2) 
= Cyt + Cayy + Cipy, + Capys + Cray + Cagye 
= Ci(yf + py, + ayi) + Colys + pus + ay2) 
=C,-0+C,-0=0. 


The proof becomes even simpler to state if we use the operator notation. An operator is 
an object that eats functions and spits out functions (kind of like what a function is, but a 
function eats numbers and spits out numbers). Define the operator L by 


Ly] = y" + py’ + ay. 


The differential equation now becomes L|y] = 0. The operator (and the equation) L being 
linear means that L[Cyy, + Coy2] = Ci L[yi|] + CoL[y2]. The proof above becomes 


Ly) = L[Ciy1 + Cayo] = Ci L[yi] + CoL[y2] = C1 -0+C2-0=0. 


Exercise 2.1.1: This fact does not hold if the equation is non-linear. Show that y;(t) = e' 


and y(t) = 1 solve 
yoaVvy-y 
but y(t) = e' + 1 does not. 


Two different solutions to the second equation y’ — k?y = 0 are y, = cosh(kx) and 
y2 = sinh(kx). Let us remind ourselves of the definition, cosh a = “*S— and sinha = =<—. 
Therefore, these are solutions by superposition as they are linear combinations of the two 
exponential solutions. 

The functions sinh and cosh are sometimes more convenient to use than the exponential. 
Let us review some of their properties: 


cosh0 = 1, sinh 0 = 0, 
— [cosh z| = sinha, au ‘sinh x] = cosh z, 
dx dx 


cosh? x — sinh? x = 1. 
Exercise 2.1.2: Derive these properties using the definitions of sinh and cosh in terms of 
exponentials. 


2.1.1 Intial Value Problems 


For first order equations, a lot of problems were stated as Initial Value Problems, containing 
both a differential equation and an initial condition of the value of y at some point x. What 
do these initial condition(s) look like for second order equations? 
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Example 2.1.1: Solve the second-order differential equation 
Y =a. 


Solution: We can attempt to find a solution to this problem by integrating both sides twice. 
A first integration gives 


and a second integration leads to 
x3 
Y= 6 + Cz + D 


for any two constants C’ and D. We can check that differentiating this y function twice gives 
us back the function x that we wanted. | 


In the previous example, we ended up with two unknown constants in our answer, whereas 
for first order equations, we only had one. In order to specify these two constants, we will 
need to give two additional facts about this function. This could be the value of the function 
at two points, but more traditionally, it is given as the value of the function y and its first 
derivative y’ at a value x9. Fairly often, this value xo is 0, but it could be any other number. 


Example 2.1.2: Solve the initial value problem 
y"=a, y(1) = 2, y(1) =3 


Solution: We previously found our solution with unknown constants as 
Pe 
= +C2-+D 


and also found that 5 


, x 
=—4+¢. 
St 


To find the values of C' and D, we need to plug in the two initial conditions into their 
corresponding functions. The initial value of the derivative gives that 
12 1 
8=y(1)=—+C=C+- 
VOUS as +5 
so that we have C' = 2. We can then use the initial value of y, along with this C’ value, to 
conclude that 


Pe G dS. 50 16 
2=yD=—+o0) tP=_4+5+D=— +P. 
Solving this out gives that D = —3 = —2. Putting these constants in gives that the solution 
to the initial value problem is 
ew 2 
year a” Sen 


6 3 | 
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For first-order equations, we have theorems that told us that solutions existed and were 
unique, at least on small intervals. Linear first-order equations in particular had a very nice 
existence and uniqueness theorem (Theorem 1.5.1), guaranteeing existence on a full interval 
wherever the coefficient functions are continuous. Linear second-order equations have an 
existence and uniqueness theorem that gives the same type of result when the initial condition 
is stated properly. 


Theorem 2.1.2 (Existence and uniqueness) 


Suppose p,q, f are continuous functions on some interval J, a is a number in J, and 
a, bg, b} are constants. The equation 


y + play’ + a(x)y = f(z), 


has exactly one solution y(xz) defined on the same interval J satisfying the initial 
conditions 


y(a) = bo, 


For example, the equation y” + k?y = 0 with y(0) = bo and y’(0) = b; has the solution 
y(x) = bo cos(ka) + 7. sin(kz). 
The equation y” — k?y = 0 with y(0) = bp and y’(0) = b, has the solution 
y(x) = bo cosh(kax) + 7! sinh( kz), 


Using cosh and sinh in this solution allows us to solve for the initial conditions in a cleaner 
way than if we have used the exponentials. 

As it did for first order equations, this theorem tells us what the proper form is for initial 
value problems for second order equations. The take-away here is that in order to fully specify 
a solution to an initial value problem, a second order equation requires two initial conditions. 
They are usually given in the form y(a) and y'(a), but could be given as y(a,) and y(a2) in 
other applications. In any case, two pieces of information are needed to determine a problem 
of second order, where we only needed one for first order. 


2.1.2 Constant Coefficient Equations - Real and Distinct Roots 


Now we want to try to solve some of these equations. As discussed earlier in this section, 
there is no explicit solution method possible for second order equations. However, if we 
restrict to a very simple case (which is also one that shows up frequently in physical systems) 
we can start to develop a method for solving these equations. The type of equation we restrict 
to is linear and constant coefficient equations. Constant coefficients means that the functions 
in front of 7", y', and y are constants, they do not depend on x. The most general second 
order, linear, constant coefficient equation is 


ay” + by’ + cy = g(x) 
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for real constants a,b,c and an arbitrary function g(x). We will study the solution of 
nonhomogeneous equations (with g(x) # 0) in § 2.5. We will first focus on finding general 
solutions to homogeneous equations, which are of the form 


ay” + by’ + cy = 0. 


Consider the problem 
y” — by’ + 8y = 0. 

This is a second order linear homogeneous equation with constant coefficients, so it fits the 
type of equation where we want to hunt for solutions. To guess a solution, think of a function 
that stays essentially the same when we differentiate it, so that we can take the function 
and its derivatives, add some multiples of these together, and end up with zero. Yes, we are 
talking about the exponential. 

Let us try* a solution of the form y = e™. Then y’ = re" and y” = re". Plug in to get 


y" — 6y' + 8y = 0, 
27r __ rx re 
ae 6re™ +8 e 0, 

vr y! y 
r?—6r+8=0 (divide through by e””), 
(r — 2)(r — 4) =0. 


Hence, if r = 2 or r = 4, then e™ is a solution. So let y; = e?” and yo = e*”. 
Exercise 2.1.3: Check that y,; and yo are solutions. 


So we have found two solutions to this differential equation! That’s great, but there may 
be a few concerning ideas at this point: 


(1) Did we just get lucky with this particular equation? 


(2) How do we know that there aren’t other solutions that aren’t of the form e’*? We made 
that assumption, so we could have missed something. 


The second point comes back to the existence and uniqueness theorem. This differential 
equation satisfies the conditions of the existence and uniqueness theorem. That means that 
as long as we find a solution that can meet any initial condition, then we know that the 
solution we have found is the only solution. We have not yet verified the part about meeting 
initial conditions yet (that’s coming later), but once we do, we’ll know that making this 
assumption is completely fine, because it got us to a solution that works, and the uniqueness 
theorem tells us that this is the only solution. 

For the first point, let’s try to generalize the calculation we did above into a method that 
will work for more equations. Suppose that we have an equation 


ay” + by’ + cy = 0, (2.3) 


*Making an educated guess with some parameters to solve for is such a central technique in differential 
equations, that people sometimes use a fancy name for such a guess: ansatz, German for “initial placement of 
a tool at a work piece.” Yes, the Germans have a word for that. 
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where a, b,c are constants. We can take our same assumption that the solution is of the form 
y = e'* to obtain 
arze™ + bre’™ + ce™ =0. 


Divide by e”” to obtain the so-called characteristic equation of the ODE: 
ar? +br+c=0. 


Solve for the r by using the quadratic formula. 


—b+ Vb? — 4ac 


2a 


1,72 = 


There are three cases that can arise based on this equation. 


(1) If b? — 4ac > 0, then we have r; and rz as two real roots to the equation. This is the 
same as the example above, and we get e”!” and e’2” as two solutions. This is the larger 
class of problems to which this exact process applies. 


(2) If b? — 4ac < 0, then r; and rg are complex numbers. We can still use e"!” and e”2” as 
solutions, but this runs into some issues, which will be addressed in Section 2.2. 


(3) If b? — 4ac = 0, then we only get one root, since r) = rz. We do get that e"” as a 


solution, but that’s all we get. This is another issue, which is addressed in Section 2.3. 


So, as long as we have b? — 4ac > 0, this method will work to give us two solutions to this 
differential equation. 


Example 2.1.3: Find two values of r so that e”™ is a solution to 
y’ + 3y' —10y = 0 


Our first step is to find the characteristic equation by plugging e”™ into the equation. This 
gives that 
r?+3r—10=0 


This polynomial factors as (r — 2)(r + 5), so we know that values of r = 2 and r = —5 will 
work. This means (check this!) that e?” and e~°” solve this differential equation. 


2.1.3. Linear Independence 


Since e?” and e~™ solve the linear differential equation in the previous example, we know 
that superposition applies, so that C,e?* + Cze~°” solves the differential equation for any 
C; and Cy. The last thing to check is that we can pick C; and C in order to meet any 
initial condition that we want. If this is possible, then we know that our method using the 
characteristic equation to find e?” and e~°* as solutions was enough to always solve this 
problem. The end of this argument is done using the existence and uniqueness theorem as 
described previously. 
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Let’s work this out. Assume that we are given bp and b; and want to solve the initial 
value problem 
y" + 3y' — 10y = 0 y(0) = bo, y/(0) = by. 


We want to do this by picking C, and C2 in the expression y = Cie?” + Cye~*” Since 
y’ = 2C,e7* — 5C2e* 
we can plug zero into this equation and the equation for y to get that we would need to have 


bo = y(0) = C1 + Ce 
by = y (0) = 2C/1 = 5C 


We can solve this system of equations by elimination. Multiplying the first equation by 5 
adding them together gives 
5b + by = TC; 
so that Shy +b 
Ci = rae 
We can then compute the value of C2 as 


5bp +b, = 2bp — By 
Cy = bo — C, = bo a aa 

Therefore, we can appropriate values of Cy and C> that will meet the initial conditions for 
arbitrary values bp and b;. This is great! This means that our method of finding solutions 
was sufficient for this problem. 

Let’s look at this situation in more generality. Assume that we have two solutions y,; and 
Ya that solve a second order linear, homogeneous differential equation, and we want to know 
if Cry, + Coyo can meet any initial condition for this problem. We have two unknowns and 
two equations (y(xo) and y’(xo) for some value x), so it should work out. 

We can carry out the same steps as above. If we have initial conditions y(a%) = bp and 
y'(xo) = b1, we want to satisfy 


bo = y(Xo) = Crys (Xo) + C2y2(xo) 
by = y' (xo) = Cry, (to) + Coyg(xo)’ 
which we get by taking the derivative of y(x) = Ciyi(x) + Coye(x) and plugging in xo. We 


will again use elimination to solve this. We can multiply the first equation by yj (ao), multiply 
the second by y;(x9), and subtract them. This will cancel out the C, term, leaving us with 


boy (0) — bryi (#0) = C2(yi (20) y2(o) — y1(%0)y5(@o))- 


We want to solve for Cy here, and once we do that, solving for C; happens by plugging back 
into one of the original equations. Most of the time, this will be completely fine, but there’s 
one issue left. We can’t divide by zero. So to be able to solve these equations for C, and Co, 
we need to know that 


Yi (0) y2(%0) — Ya(@o)yo(%o) A O. (2.4) 
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The left side of this equation is often called the Wronskian of the functions y; and yp 
at the point xo. In general, the Wronskian is the function y}(x)yo(x) — y}(x)yi(x) for two 
solutions to a second order differential equation. This relation (the Wronskian being non-zero) 
tells us that the two solutions y; and yo are different enough to allow us to meet every initial 
condition for the differential equation. This condition is so important to the study of second 
order linear equations that we give it a name. We say that two solutions y; and y2 are linearly 
independent at xo if (2.4) holds, that is, if the Wronskian of the solutions is non-zero at that 
point. For two solutions of a differential equation (which is more specific than just having two 
random functions), two solutions being linearly independent is equivalent to 2.4 holding for 
any* value 2 9 where they are defined. Our work and calculations above leads to the following 
theorem: 


Theorem 2.1.3 


Let p,q be continuous functions. Let y; and yo be two linearly independent solutions 
to the homogeneous equation (2.2). Then every other solution is of the form 


y = Cry + Coys 


for some constants C; and C . That is, y = Cyy, + Coy2 is the general solution. 


Note that this theorem works for all linear homogeneous equations, not just constant 
coefficients ones. However, the methods that we have described here (and will in future 
sections) for finding these solutions will generally only work for constant coefficient equations. 

This idea of linear independence can also be expressed in a different way: two solutions 
y, and y are linearly independent if only way to make the expression 


C1y1 + Coy2 = 0 


is by setting both c, = 0 and cp = 0. This comes from the idea of linear independence from 
linear algebra (see Chapter 3) and uniqueness of solutions to differential equations. If there 
are such constants, we can also rearrange the equation to give 


Y= -—Ye2 
Ci 


which says that y; is a constant multiple of y2, which holds for all values of x. Thus, if we 
have y; and yo, and there is no constant A so that y; = Aye, then these functions are linearly 
independent. 


Example 2.1.4: Find the general solution of the differential equation y” + y = 0. 


Solution: One of the four fundamental equations in § 0.1.4 showed that the two functions 
y. = sinx and y2 = cos” are solutions to the equation y” + y = 0. It is not hard to see 
that sine and cosine are not constant multiples of each other. If sinz = Acosx for some 
constant A, we let x = 0 and this would imply A = 0. But then sinz = 0 for all x, which 


*Abel’s Theorem, another theoretical result, says that the Wronskian yj y2 — yi y4 is either always zero 
or never zero. That means that any one value can be checked to determine if two solutions are linearly 
independent. Picking 0 is usually a convenient choice. 


120 CHAPTER 2. HIGHER ORDER LINEAR ODES 


is preposterous. So y; and y2 are linearly independent. We could also have checked this by 
taking derivatives and plugging in zero. Since 


yi(0) =0 y(0)=1 y2(0)=1 y,(0) =0 
we have that 


Yi, (0)y2(0) — yr (0)yo(0) = (1)(1) — (0)(0) = 1 40 


so these solutions are linearly independent. Hence, 
y = Ci cosx + Co sin x 


is the general solution to y” + y = 0. | 


For two functions, checking linear independence is rather simple. Let us see another 
example using non-constant coefficient equations. Consider y” — 2x-2y = 0. Then y; = 2? 
and yg = !/z are solutions. To see that they are linearly indepedent, suppose one is a multple 
of the other: y; = Ay2, we just have to find out that A cannot be a constant. In this case we 
have A = %/y. = x, this most decidedly not a constant. So y = C,x? + C!/z is the general 
solution. 


Now, back to our discussion of constant coefficient equations. If b? — 4ac > 0, then we 
have two distinct real roots r; and rg, giving rise to solutions of the form y;(x) = e™ and 
Y2(x) = e™*. Using condition 2.4 with rp = 0, we compute 


Yi (0)y2(0) — y1(0)yo(0) = (ri)(L) — (1)(r2) = 1 — 12. 


Since r; # ro, this expression is not zero, so the two solutions are linearly independent. 
Therefore, in this case, we know that the general solution will be 


y= Ce" + Coe™. 


Using the other formulation of linear independence of two functions, we would need to show 
that there is no constant A so that 


el? = Aer, 


Since this can be rewritten as A = e'2-")* and we know that rj # ro, this is not a constant, 
so we again know that these functions are linearly independent and give rise to a general 
solution. 


Example 2.1.5: Solve the initial value problem 
y+ 2y—3y=0 = y(0) =2, yO) = 1. 


Solution: To start, we find the characteristic equation of this differential equation and look 
for the roots. The characteristic equation here is 


r?+2r—3=0 
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and this factors as (r + 3)(r — 1) =0. Thus, the two roots are r = 1 and r = —3, so that the 
general solution (and we know it is the general solution because these are different exponents 
and so the solutions are linearly independent) is 


y(@) =Cye” + Cre ™. 


In order to find the values of C; and Cy, we need to use the initial conditions. Plugging 
zero into y(x) gives 


and since the derivative y'(x) = Ce” — 3Cze~”, the second condition gives that 
HOSTS On Be. 
Subtracting the second equation from the first gives that 
1=4C, 
so that Cp = 1/4 and C, = 7/4. Thus, the solution to the initial value problem is 


7 1 
y(t) Se" +e. 
4 4 
=| 


In this second example, we solve a problem in the same way, but the roots of the 
characteristic equation do not work out as nicely. Even with that, the structure and process 
for the problem is identical to the previous example. 


Example 2.1.6: Solve the initial value problem 


y" —2y-y=0 = y(0) = 2, y(0) =3. 


Solution: We start by looking for the characteristic equation of this differential equation 
and finding its roots. The characteristic equation is 


eae 


which has roots 


= 2+ ,/(—2)? — 4(1)(-1) _ 2+ V8 eee. 


r 


2 2 
There are two real and distinct roots, so we know that the two solutions y,(2) = e@+v?) 
and y2(x) = e"-V?)* are linearly independent, so we have that the general solution to this 


problem is 
yl) = CyetV* + Cycle, 


Next, we need to find the constants C; and C2 to meet the initial conditions. We can see 
that, by computing the first derivative, 


y(w) = CyetV2)" 4 Cye@-v22, 
y (x) =(1+ W2\Cyeo Vv” +(1- V2)\Cx30-V?*, 
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and plugging in x = 0 gives that we want C; and C2 to solve 
2=C,4+C, 
3 = (14+ V2)C, + (1— V2)C. 


We can solve this by any method. One trick at the start is to subtract equation 1 from 
equation 2, giving that 


P= OO, 
1 = V2C, — V2Cy, 


which can be rewritten as 


Adding these equations together and dividing by 2 gives that 


1 
2C, = 2+ — 
1 5) 


so that Cy = 1+ 575) and since C; + Cy = 2, we have that Cy) = 1 — aE Therefore, the 


solution to the desired initial value problem is 


1 1 
Aa (pct eens p (: _ lanes 
ao) ( 7A) D4 9 


2.1.4 Exercises 

Exercise 2.1.4: Show that y = e® and y = e** are linearly independent. 
Exercise 2.1.5:* Are sin(xz) and e” linearly independent? Justify. 
Exercise 2.1.6:* Are e® and e**? linearly independent? Justify. 
Exercise 2.1.7:* Guess a solution to y" + y' +y =5. 

Exercise 2.1.8: Take y” + 5y = 10x +5. Find (guess!) a solution. 


Exercise 2.1.9: Verify that y(t) = e' cos(2t) and yo(t) = e' sin(2t) both solve y” —2y'+5y = 
0. Are these two solutions linearly independent? What does that mean about the general 
solution to y” — 2y' + 5y = 0? 


Exercise 2.1.10: Prove the superposition principle for nonhomogeneous equations. Suppose 
that y; is a solution to Ly, = f(x) and yp is a solution to Ly = g(x) (same linear operator 
L). Show that y = y1 + ye solves Ly = f(x) + g(a). 
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Exercise 2.1.11: Determine the maximal interval of existence of the solution to the differ- 


ential equation 
1 cos(t) 
t—5 " / _—— 
CO ie ge ed ag 
with initial condition y(3) = 8. What about if the initial condition is y(—3) = 4? 


Exercise 2.1.12: For the equation xy" — xy’ = 0, find two solutions, show that they are 
linearly independent and find the general solution. Hint: Try y = x". 

Exercise 2.1.13:* Find the general solution to ry" + y' = 0. Hint: It is a first order ODE 
Thy; 

Exercise 2.1.14: Find the general solution of 2y" + 2y' — 4y = 0. 

Exercise 2.1.15: Solve y"” + 9y' = 0 with y(0) = 1, y’(0) =1. 

Exercise 2.1.16: Find the general solution of y” + 9y' — 10y = 0. 

Exercise 2.1.17: Find the general solution to y” — 3y' — 4y = 0. 

Exercise 2.1.18: Find the general solution to y" + 6y' + 8y = 0. 

Exercise 2.1.19: Find the solution to y” — 3y' + 2y = 0 with y(0) = 3 and y’(0) = —1. 
Exercise 2.1.20: Find the solution to y” + y’ — 12y = 0 with y(0) = 1 and y'(0) = —2. 
Exercise 2.1.21:* Find the general solution to y"” + 4y' + 2y = 0. 

Exercise 2.1.22:* Find the solution to 2y" + y' — 3y = 0, y(0) =a, y'(0) = b. 


Exercise 2.1.23:* Find the solution to y” —(a+f)y'+aBy = 0, y(0) =a, y'(0) = b, where 
a, 8, a, and b are real numbers, and a F B. 

Exercise 2.1.24:* Write down an equation (guess) for which we have the solutions e* and 
e**, Hint: Try an equation of the form y” + Ay' + By = 0 for constants A and B, plug in 
both e” and e*” and solve for A and B. 

Exercise 2.1.25:* Construct an equation such that y = C,e®” + Coe 
solution. 


2x is the general 


Exercise 2.1.26: Give an example of a 2nd-order DE whose general solution is y = cje~7! + 
dy 
C2e . 


Equations of the form ax7y”" + bry’ + cy = 0 are called Euler’s equations or Cauchy—Euler 
equations. They are solved by trying y = 2” and solving for r (assume that x > 0 for 
simplicity). 

Exercise 2.1.27: Suppose that (b — a)? — 4ac > 0. 
a) Find a formula for the general solution of ax*y" + bry’ + cy = 0. Hint: Try y = x" and 
find a formula for r. 
b) What happens when (b — a)? — 4ac = 0 or (b— a)? — 4ac < 0? 
We will revisit the case when (b — a)? — dac < 0 later. 


Exercise 2.1.28: Same equation as in Exercise 2.1.27. Suppose (b — a)” — 4ac = 0. Find 
a formula for the general solution of ax*y" + bry’ + cy = 0. Hint: Try y = x" nx for the 
second solution. 
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2.2 Complex Roots and Euler’s Formula 


Attribution: [JL], §2.2. 


After this section, you will be able to: 


e Understand the basics of complex numbers, 


e Use complex numbers to find complex solutions to second order constant coefficient 
equations, and 


e Use Euler’s formula to find real-valued general solutions to these second order 
equations. 


The next case to consider for constant coefficient second order equations is the one where 
b? — 4ac < 0. This results in two roots 7; and rz, but they are complex roots. In order to 
solve differential equations with b? — 4ac < 0, we need to be able to manipulate and use some 
properties of complex numbers. Complex numbers may seem a strange concept, especially 
because of the terminology. There is nothing imaginary or really complicated about complex 
numbers. For more bachground information on complex numbers, see Appendix B.2. 

To start with, we define i = /—1. Since this is the square root of a negative number, this 
7 is not a real number. A complex number is written in the form z = x + iy where x and y 
are real numbers. For a complex number x + zy we call x the real part and y the imaginary 
part of the number. Often the following notation is used, 


Re(z + iy) = 2 and Im(z + iy) = y. 


The real numbers are contained in the complex numbers as those complex numbers with the 
imaginary part being zero. 

When trying to do arithmetic with complex numbers, we treat 7 as though it is a variable, 
and do computations just as we would with polynomials. The important fact that we will 
use to simplify is the fact that since 1 = /—1, we have that i? = —1. So whenever we see 7”, 
we replace it by —1. For example, 


(2 + 3i)(4i) — bi = (2 x 4)i + (3 x 4)? — 5a = 81 + 12(—1) — Bi = —12 + 334. 


The numbers i and —i are the two roots of r? + 1=0. Engineers often use the letter j 
instead of 2 for the square root of —1. We use the mathematicians’ convention and use 7. 


Exercise 2.2.1: Make sure you understand (that you can justify) the following identities: 
1 
a) 2 =—1, 8 =i, i =1, ao 
i 


c) (3 — Ti)(—2 — 91) =--- = —69 — 134, d) (3—2i)(3+2i) = 3?—(2i)? = 3242? = 13, 


aaa ae ae ee ae 
ya Sa B Bt 7B 
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In order to solve differential equations where the characteristic equation has complex 
roots, we need to deal with the exponential e*+” of complex numbers. We do this by writing 
down the Taylor series and plugging in the complex number. Because most properties of the 
exponential can be proved by looking at the Taylor series, these properties still hold for the 
complex exponential. For example the very important property: e*tY = e*e’. This means 
that e*+”” = e%e". Hence if we can compute e”, we can compute e**”. For e”’, we use the 
so-called Euler’s formula. 


Theorem 2.2.1 (Euler’s formula) 


e”” = cos6 +isin@ and e~” = cos@ — isin @. 


In other words, e**”? = e*(cos(b) + isin(b)) = e* cos(b) + ie* sin(b). 


Exercise 2.2.2: Using Euler’s formula, check the identities: 


ci9 4 pid eid ~i6 
cos @ = = and sind = 


Exercise 2.2.3: Double angle identities: Start with e’?® = (ci”)’. Use Euler on each side 
and deduce: 


cos(20) = cos” @ — sin? 0 and sin(20) = 2sin @ cos @. 


2.2.1 Complex roots 


Suppose the equation ay” + by’ + cy = 0 has the characteristic equation ar? + br +c = 0 
that has complex roots. By the quadratic formula, the roots are abv ba dae These roots are 
complex if b? — 4ac < 0. In this case the roots are 


—b  .V4ac — b? 
T7172 = +4 : 
2a 2a 


As you can see, we always get a pair of roots of the form a +i(. In this case we can still 
write the solution as 


However, the exponential is now complex-valued, and so (real) linear combinations of these 
solutions will be complex valued. If we are using these equations to model physical problems, 
the answer should be real-valued, as the position of a mass-on-a-spring can not be a complex 
number. To do this, we need to determine two real-valued, linearly independent solutions to 
this differential equation. 

To do this, we use the following result. 
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Theorem 2.2.2 


Consider the differential equation 


y” + p(z)y’ + q(x)y = 0 


where p(t) and q(t) are real-valued continuous functions on some interval J. If y 
is a complex-valued solution to this differential equation and we can split y(x) = 
u(x) + tv(x) into its real and imaginary parts u and v, then u and v are both solutions 


toy" + p(x)y' + g(x)y = 0. 


Proof. This is based on the fact that the differential equation is linear. We can compute 
derivatives of y 


y(2) = u(x) + to(x) 
y'(a) = u(x) + iv'(x) 
yf (x) = u"(a) + w(x) 


Then, we can plug this into the differential equation 
O= y" + plx)y' + alx)y 


ul (a) + iv"(x) + p(x)(u'(x) + tv"(x)) + g(@) (u(x) + iv(a)) 
0 = ul(x) + p(a)u'(x) + q(@)u(@) + i(u"(w) + plw)o'(a) + a(a)o(@)) 


Since the equation at the end of this chain is equal to zero, it must be zero as a complex 
number, which means that both the real and imaginary parts must be zero. This means that 


u(x) + p(x)u'(2) + 9(@)u(x) = 
v(x) + p(a)v'(x) + q(2)o(x) = 


so that both u and v solve the original differential equation. 


To use this to solve the problem at hand, we have our solution 
yi(z) = enti 
and we need to split this into its real and imaginary parts. Since 


yi = e™ cos(Bx) + ie™ sin( Bx), 


the real and imaginary parts of this function are 


which, by the previous theorem, we know are also solutions. These are two solutions to our 
original differential equation that are also real-valued! 
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On the other hand, assume that we take the other complex solution, which will be 


a—iba 


y(t) =e 
If we split this into real and imaginary parts, we will get 
yo = e* cos(Bx) — ie sin(6z), 


so that the real and imaginary parts of this solution are 


These are exactly the same as the previous real and imaginary parts, up to the minus sign on 
Ug. Since we are going to incorporate these with constants C, and C2 eventually, they will 
give rise to the same general solution. So, we only need one of these two complex solutions 
to generate our two linearly independent real-valued solutions, and either of the two complex 
solutions give the same pair of real-valued solutions. 


Exercise 2.2.4: For 6 # 0, check that e® cos(3x) and e®* sin(Gx) are linearly independent. 


With that fact, we have the following theorem. 
Theorem 2.2.3 


Take the equation 
ay” + by’ + cy = 0. 


If the characteristic equation has the roots a +73 (when b? —4ac < 0), then the general 
solution is 


y = Cie” cos( 8x) + Coe™ sin( Bz). 


Example 2.2.1: Find the general solution of y” + k?y = 0, for a constant k > 0. 
Solution: The characteristic equation is r? + k? = 0. Therefore, the roots are r = +ik, and 
by the theorem, we have the general solution 
y = Cicos(kx) + Co sin(kz). 
_ 
Example 2.2.2: Find the solution of y” — 6y’ + 13y = 0, y(0) = 0, y’(0) = 10. 


Solution: The characteristic equation is r? — 6r + 13 = 0. By completing the square we get 
(r — 3)” + 2? = 0 and hence the roots are r = 3+ 2i. By the theorem we have the general 
solution 

y = Cye* cos(2r) + Cye** sin(2z). 


To find the solution satisfying the initial conditions, we first plug in zero to get 


0 = y(0) = Cie’ cos0 + Coe® sin 0 = Ch. 
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Hence, C; = 0 and y = Cye** sin(2r). We differentiate, 
y’ = 3Cre** sin(2xr) + 2C2e** cos(2z). 


We again plug in the initial condition and obtain 10 = y’(0) = 2C2, or Cj = 5. The solution 
we are seeking is 

y = 5e* sin(2z). 

_ 

In this previous example, we can get a fairly good idea of how to sketch out the graph of 

this function. Since sin(2z) oscillates between —1 and 1, the graph of y = 5e*” sin(2z) will 

oscillate between the graphs of 5e®* and —5e®”. These curves that surround the graph of 

the solution are called envelope curves for the solution. In Figure 2.1, this phenomenon is 

illustrated for the function y = 2e” sin(5z). 


Figure 2.1: Plot of the function y = 2e” sin(5x) with envelope curves. 


This is simple when there is only one term in the function we want to draw. When both 
sine and cosine terms appear, this can get more tricky, but we can still work it out. In the 
more general case, the solution will look something like 


y = Ae™ cos(8x) + Be™ sin(Gzx). 


We can first factor out an e®°”, and then we want to write Acos(3x) + Bsin(Gx) as a single 
trigonometric function. The identity we want to use here is the trigonometric identity 


cos(3a — 6) = cos(d) cos(Gx) + sin(d) sin(Gz). 
If there is an angle 6 so that A = cos(d) and B = sin(d), then we could write 
Acos(Gx) + Bsin(6x) = cos(Sx — 6) 


and we would be done. However, this does not always happen; the main issue being that 
cos*(6) + sin?(6) = 1 for all 6, but it is not necessarily the case that A? + B? = 1. But we can 
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force this last condition. If we define R = VA? + B?, then we can rewrite this expression as 


cos(Bx) + sn( 0) 


Acos(3x) + Bsin(Gx) = R (= a 
= R(cos(d) cos(Gx) + sin(d) sin(3z)) 
= Rcos(Gx — 0) 


where 0 is the angle so that 


cos(6) = “ sin(d) = > 


and such an angle will always exist. Therefore, we can represent the original solution 


y = Ae cos(8x) + Be™ sin(Gx) 


as 
y = Re™ cos(Gba — 6) 
where ri B 
RSV Ae Be? cos(d) = R sin(d) = Rp 
Therefore, the envelope curves for this solution will be 


ySthe™, 


Note that in order to determine these envelope curves, you do not need to determine the 
6 value in the representation of the solution. All you need is the value of R, which can be 
computed as A? + B? where A and B are the coefficients of the sine and cosine terms in 
the solution. 


Example 2.2.3: Find the solution to the initial value problem 
y" + 2y'+5y=0 (0) =1, y/(0) =5. 


Determine a value T’ where the solution y(z) satisfies |y(x)| < 0.1 for alla > T. 


Solution: We solve the initial value problem by normal techniques from this section. The 
characteristic equation is r? + 2r +5 = 0, which has roots r = —1 +2. Therefore, the general 
solution of the differential equation is 


y = Cie * cos(2x) + Coe sin(22). 
Plugging in 0 gives that y(0) = 1 = C), and the derivative of this general solution is 
y = —Cye-* cos(2x) — 2C,e~” sin(2x) — Cye~* sin(2x) + 2Cye~* cos(2z). 


Plugging in 0 here gives 
y'(0) = -—C; + 2C. 
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Since C; = 1, this gives that C2) = 3. So, our solution is 
y(x) =e * cos(2x) + 3e~* sin(2z). 


Through the work above, we can find R= /14+9 =v 10. Therefore, the envelope curves 
for the solution are 
+V10e~”. 


In order to find this threshold 7 where the solution will stay within 0.1 of zero, we need to 
figure out when this envelope curves get to the 0.1 threshold. Once the envelope curves get 
to that level, we know that the full solution must be trapped there as well. We can solve 


0.1 
0.1 = V10e-7 T=-l1n (=) we 3.454. 
/10 


So, for all values of x larger than 3.454, the solution will be within 0.1 of zero. This is 
illustrated in Figure 2.2. Note that we did not find the best value T here, as it probably 
could be made smaller using the actual solution. The issue here is that because the solution 
is oscillating, it may end up staying inside the 0.1 cutoff before that value of time, but this is 
the lowest value of T’ that we can prove and validate using envelope curves. | 


Figure 2.2: Plot of the function e~* cos(2x) + 3e~* sin(2x) with envelope curves illustrating the 
bounds on the function for large values of x. 


2.2.2 Exercises 

Exercise 2.2.5:* Write 3cos(2x) + 3sin(2x) in the form Rcos(Gx — 6). 
Exercise 2.2.6: Write 2cos(3x) + sin(3x) in the form Rcos(Gx — 6). 
Exercise 2.2.7: Write 3cos(x) — 4sin(x) in the form Rcos(Gx — 0). 


Exercise 2.2.8: Show that e?* cos(x) and e”” sin(x) are linearly independent. 
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Exercise 2.2.9: Find the general solution of 2y” + 50y = 0. 

Exercise 2.2.10: Find the general solution of y” — 6y' + 13y = 0. 

Exercise 2.2.11: Find the solution to y” — 2y' + 5y = 0 with y(0) = 8 and y'(0) = 2. 
Exercise 2.2.12: Find the general solution of y” + 2y' — 3y = 0. 

Exercise 2.2.13:* Find the solution to 2y” + y' + y = 0, y(0) = 1, y’(0) = —2. 
Exercise 2.2.14:* Find the solution to z"(t) = —2z'(t) — 2z(t), z(0) = 2, z'(0) = —2. 


Exercise 2.2.15: Let us revisit the Cauchy—Euler equations of Exercise 2.1.27 on page 123. 
Suppose now that (b— a)? — 4ac < 0. Find a formula for the general solution of ax?y! + 
bry’ + cy = 0. Hint: Note that 2” = e™™*. 


Exercise 2.2.16: Construct an equation such that y = C,e~” cos(3x) + Cpe~” sin(32) is 
the general solution. 


Exercise 2.2.17:* Find a second order, constant coefficient differential equation with general 
solution given by y(t) = Cye” cos(2x) + Ce?” sin(x) or explain why there is no such thing. 


Exercise 2.2.18: Find a second order, constant coefficient differential equation with general 
solution given by y(t) = C\e* cos(2x) + Ce” sin(2x) or explain why there is no such thing. 


Exercise 2.2.19: Find the solution to the initial value problem 
y+4y'+5y=0 y(0) =3, y/(0) =-1. 

Determine a value T so that |y(x)| < 0.02 for alla > T. 

Exercise 2.2.20: Find the solution to the initial value problem 
yf’ +6y'+138y=0 y(0)=4, y/(0) =7. 


Determine a value T so that |y(x)| < 0.01 for alla > T. 
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2.3. Repeated Roots and Reduction of Order 


Attribution: [JL], §2.1, 2.2. 


After this section, you will be able to: 


e Find the general solution to a second order constant coefficient equation with 


repeated roots, 

e Apply the method of reduction of order to generate a second solution to an 
equation given one solution, and 

e Solve Euler equations using the method of reduction of order. 


The last case we have to handle for solving all second order linear constant coefficient 
equations is the case where b? — 4ac = 0 in the equation 


ay” + by’ + cy = 0. 


When we try to find the characteristic equation and find solutions to this equation, we get a 
double root at r;, so that the characteristic polynomial is (r — r,)?. For this, we get that 
e™* is a solution. However, that’s the only solution we get. We need to have two linearly 
independent solutions in order to get the general solution to the differential equation, so we 
need to find some method to get another solution. The standard method, and the one we 
apply here is reduction of order. Let’s see how this works through an example. 


Example 2.3.1: Find two linearly independent solutions to the differential equation 
y+ 2y' +y = 0. 


Solution: To start, we find the first solution using our original method. The characteristic 
equation here is r? + 2r + 1 = 0, which is (r + 1)?. Therefore, we have a double root at 
r = —1, so that y|(x) =e” is a solution. 

To find a second solution, the reduction of order method suggests that we try to plug in 
y = v(x)e* for an unknown function v(x). The goal is to figure out an equation that v must 
satisfy to see if this leads us to a second solution to the original equation. We can compute 
the first two derivatives of y = u(x)e-* 


u(xa)e* 
y (x) =v'(x)e"” — v(z)e 
vu" (x)e* — 2u'(x)e* + v(a)e* 


—2£ 


and then plug them into the original differential equation 


O=y" + 2y'+y 
= (v"(x)e~* — Qv'(x)e~* + v(a)e"”) + 2(u'(xz)e — v(a)e~*) + v(x)e~* 
=v" (rje* +u'(x)(—2e°* + 2e°-*) + u(x) (e* — 20 * +e”) 


—2x 


= wv" (ale 
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Since e~* is never zero, this means we must have v"(x) = 0. This is still a second order 


equation, but we know how to solve it. We can integrate both sides twice to get that 
u(x) = Ax + B for any constants A and B. 

Our goal with all of this was to find a solution y of the form u(xz)e~*. The set up here 
means that y = (Az + B)e~* will solve the differential equation. Since we already knew that 
Be~* was a solution, the new information we gained here was that Axe~”, or in particular, 
xe* is a solution to the differential equation. Thus, our two solutions are y;(x) = e~* and 


(ole wer: | 


Exercise 2.3.1: Check that e~* and xe~* both solve y” + 2y' + y = 0, and that these 
solutions are linearly independent. 


The reduction of order method applies more generally to any second order linear homoge- 
nous equation and the goal is the same: use one solution of the differential equation to generate 
another one. The idea is that if we somehow found y; as a solution of y” + p(x)y’ + q(x)y = 0 
we try a second solution of the form y2(x) = y1(x)v(x). We just need to find v. We plug y2 
into the equation: 


0 = yy + p(x)y + a(@) yo = yu + 2yyu' + yo” + p(x) (yu + yw’) + q(z)yiv 
0 


= yu" + (2y, + p(x)y)o' + (yf + pay o(x)y1) 0. 


In other words, y,v" + (2y; + p(x)yi)u’ = 0. Using w = v' we have the first order linear 
equation yw’ + (2y; + p(x)y1)w = 0. After solving this equation for w (integrating factor), 
we find v by antidifferentiating w. We then form yz by computing y,v. For example, suppose 


we somehow know y; = x is a solution to y” + 27!y’ — 2~2y = 0. The equation for w is then 


cw’ + 3w =0. We find a solution, w = Cx~3, and we find an antiderivative v = =o. Hence 
Yo=Yv= re Any C works and so C = —2 makes yp = 1/z. Thus, the general solution is 


y = Cyr + Co1/z. 

The easiest way to work out these problems is to remember that we need to try yo(x) = 
yi(x)vu(x) and find u(x) as we did above. Also, the technique works for higher order equations 
too: you get to reduce the order for each solution you find. 

In summary, for constant coefficient equations with a repeated root, the reduction of 
order method will always give the equation v” = 0, and so the solution is v(x) = Ax + B. 
Multiplying by the y; solution e”™ gives that xe’ is the other solution. Therefore, the general 
solution for repeated root equations is always of the form 


Y= Cie" + Cyxr™”, 
Example 2.3.2: Find the general solution of 
y — 8y' + 16y = 0. 


Solution: The characteristic equation is r? — 8r + 16 = (r — 4)? = 0. The equation has a 
double root r; = rg = 4. The general solution is, therefore, 


y=(Ci+ Coz)e™ = Cie™* + Core™. 
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Exercise 2.3.2: Check that e*” and xe*” are linearly independent. 


That e** solves the equation is clear. If xe*” solves the equation, then we know we are 
done. Let us compute y’ = e* + 4ve*” and y” = 8e* + 16xre*”. Plug in 


yf" — 8y' + 16y = 8e™ + 16ze*” — 8(e* + 4ze*”) + 16re*" = 0. 


= 


In some sense, a doubled root rarely happens. If coefficients are picked randomly, a 
doubled root is unlikely. There are, however, some natural phenomena where a doubled root 
does happen, so we cannot just dismiss this case. In addition, there are specific physical 
applications that involve the double root problem, which we will discuss in Section 2.4. 
Finally, the solution with a doubled root can be thought of as an approximation of the 
solution with two roots that are very close together, and the behavior of this solution will 
approximate “nearby” solutions as well. 


Example 2.3.3: Find the solution y(t) to the initial value problem 
y’ +6y +9y=0 y(0)=2, y'(0) = —-3. 
Solution: The characteristic polynomials for this differential equation is 
r?+6r+9 


which factors as (r +3)”, so that we have a double root at —3. With the work done previously, 
we know that the general solution is 


y(t) = (Cy + Cote = Cie + Cote. 
If we use the initial conditions, we can set t = 0 to get that 
2=y(0) =Cie° 
so that C, = 2. Differentiating the general solution gives that 
yf (£) = —380,6 * + Coe ** — 3Cate * 
and plugging in zero here gives 
—3=y'(0) = -38C, + Co. 
Since C; = 2, this implies that C2 = 3. Therefore, the solution to this initial value problem is 


y(t) = 2e7* + 3te~™. 
sz 
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2.3.1 Exercises 

Exercise 2.3.3: Find the general solution to y" + 4y' + 4y = 0. 

Exercise 2.3.4:* Find the general solution to y"” — 6y' + 9y = 0. 

Exercise 2.3.5: Find the solution to y” + 6y' + 9y = 0 with y(0) = 3 and y’(0) = —1. 
Exercise 2.3.6: Solve y" — 8y' + 16y = 0 for y(0) = 2, y’(0) = 0. 

Exercise 2.3.7: Find the general solution of y” = 0 using the methods of this section. 


Exercise 2.3.8: The method of this section applies to equations of other orders than two. 
We will see higher orders later. Try to solve the first order equation 2y' + 3y = 0 using the 
methods of this section. 


Exercise 2.3.9: Consider the second-order DE 
ty” + (4t + 2)y’ + (4¢+ 4)y = 0. (2.5) 


a) Does the superposition principle apply to this DE? Give a one- or two-sentence expla- 
nation wither way. 


b) Find a value of r so that y = e™ is a solution to (2.5) 


c) Using your result from the previous page, apply reduction of order to find the general 
solution to (2.5). 


Exercise 2.3.10 (Euler Equations):* Consider the differential equation x*y" +3xry'—3y = 0. 
a) Verify that y:(x) = x is a solution. 
b) Use reduction of order to find a second linearly independent solution. 
c) Write out the general solution. 

Exercise 2.3.11 (Euler Equations):* Consider the differential equation x*y" +4axy'—2y = 0. 
a) Verify that y;(x) = + is a solution. 
b) Use reduction of order to find a second linearly independent solution. 
c) Write out the general solution. 

Exercise 2.3.12 (Euler Equations):* Consider the differential equation xy" —6xy'—10y = 0. 
a) Verify that y,(x) = x? is a solution. 
b) Use reduction of order to find a second linearly independent solution. 


c) Write out the general solution. 
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Exercise 2.3.13: Write down a differential equation with general solution y = at? + bt~°, 
or explain why there is no such thing. 


Exercise 2.3.14: Find the solution to y" — (2a)y’ + a®y = 0, y(0) =a, y'(0) = b, where a, 
a, and b are real numbers. 


Exercise 2.3.15 (reduction of order): Suppose y; is a solution to y” + p(x)y' + q(x)y = 0. 
By directly plugging into the equation, show that 


en J p(x) dex 
n(e) = nla) | de 
(v(x) 
is also a solution. 
Exercise 2.3.16 (Chebyshev’s equation of order 1): Take (1 — x2”)y” — xy’ + y = 0. 
a) Show that y = x is a solution. 
b) Use reduction of order to find a second linearly independent solution. 
c) Write down the general solution. 
Exercise 2.3.17 (Hermite’s equation of order 2): Take y” — 2ry' + 4y = 0. 


a) Show that y = 1 — 22? is a solution. 


b) Use reduction of order to find a second linearly independent solution. (It’s OK to leave 
a definite integral in the formula.) 


c) Write down the general solution. 


The rest of these exercises can be solved using any of the methods discussed in the last three 
sections. Pick the appropriate method in order to solve the problem. 


Exercise 2.3.18: Find the general solution of y” + 5y' — 6y = 0. 


Exercise 2.3.19: Find the general solution of y” — 2y' + 2y = 0. 


Exercise 2.3.20: Find the general solution of y" + 4y' + 4y = 0. 


Exercise 2.3.21: Find the general solution of y" + 4y' + 5y = 0. 


Exercise 2.3.22: Find the solution to y” — 6y' + 13y = 0 with y(0) = 2 and y’(0) = 1. 
Exercise 2.3.23: Find the solution to y” + 4y’ — 12y = 0 with y(0) = —1 and y/(0) = 3. 


Exercise 2.3.24: Find the solution to y” — 6y' + 9y = 0 with y(0) = —4 and y'(0) = —1. 
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2.4 Mechanical vibrations 


Attribution: [JL], §2.4. 


After this section, you will be able to: 


e Write second-order differential equations to model physical situations, 


e Classify a mechanical oscillation as undamped, underdamped, critically damped, 
or overdamped, and 

e Use the solution to a differential equation to describe the resulting physical 
motion. 


In the last few sections, we have discussed all of the different possible solutions to 
constant coefficient second order differential equations, whether the roots of the characteristic 
polynomial real and distinct, complex, or repeated. Now, we want to look at applications of 
these equations, now that we know how to solve them. Since Newton’s Second Law F’ = ma 
involves the second derivative of position (acceleration), it is reasonable that a lot of physical 
systems will be defined by second order differential equations. 

Our first example is a mass on a spring. Suppose we have a k F(t) 
mass m > 0 (in kilograms) connected by a spring with spring m |» 
constant k > 0 (in newtons per meter) to a fixed wall. There 
may be some external force F(t) (in newtons) acting on the 
mass. Finally, there is some friction measured by c > 0 (in 
newton-seconds per meter) as the mass slides along the floor (or perhaps a damper is 
connected). 

Let x be the displacement of the mass (x = 0 is the rest position), with x growing to the 
right (away from the wall). The force exerted by the spring is proportional to the compression 
of the spring by Hooke’s law. Therefore, it is kx in the negative direction. Similarly the 
amount of force exerted by friction is proportional to the velocity of the mass. By Newton’s 
second law we know that force equals mass times acceleration and hence ma” = F'(t)—ca!—ka 
or 


damping c 


mx" + ca'+kax = F(t). 
This is a linear second order constant coefficient ODE. We say the motion is 
(i) forced, if F £0 (if F is not identically zero), 
(ii) unforced or free, if F = 0 (if F is identically zero), 
(iii) damped, if c > 0, and 
) 


(iv) undamped, if c = 0. 


This system appears in lots of applications even if it does not at first seem like it. Many 
real-world scenarios can be simplified to a mass on a spring. For example, a bungee jump 
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setup is essentially a mass and spring system (you are the mass). It would be good if someone 
did the math before you jump off the bridge, right? Let us give two other examples. 


Here is an example for electrical engineers. Consider the pictured 


II 
RLC circuit. There is a resistor with a resistance of R ohms, an inductor ©) C 
with an inductance of L henries, and a capacitor with a capacitance R L 
of C' farads. There is also an electric source (such as a battery) giving ==—_~a,a,A,—_ 


a voltage of E(t) volts at time t (measured in seconds). Let Q(t) be 

the charge in coulombs on the capacitor and /(t) be the current in the circuit. The relation 
between the two is Q’ = I. By elementary principles we find LI’ + RI + @/c = E. Since 
QQ’ = 1, this means that J’ = Q”, and we can write this equation as 


LQ"(t) + RQ'(t) + Z(t) = El). 


We can also write this a different way by differentiating the entire equation in t to get a 
second order equation for I(t): 


LI"(t) + RI'(t) + al(t) ~ B'(t). 


This is a nonhomogeneous second order constant coefficient linear equation. As L, R, and 
C are all positive, this system behaves just like the mass and spring system. Position of 
the mass is replaced by current. Mass is replaced by inductance, damping is replaced by 
resistance, and the spring constant is replaced by one over the capacitance. The change in 
voltage becomes the forcing function—for constant voltage this is an unforced motion. 


Our next example behaves like a mass and spring system only 
approximately. Suppose a mass m hangs on a pendulum of length 
L. We seek an equation for the angle 6(t) (in radians). Let g 
be the force of gravity. Elementary physics mandates that the 
equation is 


OU + 4 sind = (). 


Let us derive this equation using Newton’s second law: force 
equals mass times acceleration. The acceleration is LO” and mass ‘ 
is m. So mL6” has to be equal to the tangential component of the force given by the gravity, 
which is mg sin @ in the opposite direction. So mL@é” = —mgsin@. The m curiously cancels 
from the equation. 

Now we make our approximation. For small 6 we have that approximately sin@ = @. This 
can be seen by looking at the graph. In Figure 2.3 on the facing page we can see that for 
approximately —0.5 < @ < 0.5 (in radians) the graphs of sin @ and @ are almost the same. 

Therefore, when the swings are small, 6 is small and we can model the behavior by the 
simpler linear equation 


gl" + aa =f), 


The errors from this approximation build up. So after a long time, the state of the real-world 
system might be substantially different from our solution. Also we will see that in a mass- 
spring system, the amplitude is independent of the period. This is not true for a pendulum. 
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Nevertheless, for reasonably short periods of time and small swings (that is, only small angles 
0), the approximation is reasonably good. 

In real-world problems it is often necessary 
to make these types of simplifications. We vo f z es 
must understand both the mathematics and 
the physics of the situation to see if the simpli- 
fication is valid in the context of the questions 
we are trying to answer. 


2.4.1 Free undamped motion 


In this section we only consider free or unforced 
motion, as we do not know yet how to solve 4. | | ; 40 
nonhomogeneous equations. Let us start with = a YS * = 
undamped motion where c= 0. The equation — Figure 2.3: The graphs of sin@ and 6 (in 
is radians). 

ma” + ka = 0. 


We divide by m and let wo = \/*/m to rewrite the equation as 


a" + wee =0. 
The general solution to this equation is 
x(t) = Acos(wot) + Bsin(wot). 
By a trigonometric identity that we discussed previously in § 2.2, 
Acos(wot) + Bsin(wot) = C cos(wot — 6), 


for two constants C' and y. Earlier, we found that we can compute these constants as 
C = VA? + B? and tand = 8/4. Therefore, we let C and 6 be our arbitrary constants and 
write x(t) = C'cos(wot — 6). 


Exercise 2.4.1: Justify the identity Acos(wot) + Bsin(wot) = Ccos(wot — 6) and verify 
the equations for C' and 6. Hint: Start with cos(a — (3) = cos(a@) cos(3) + sin(a@) sin(G) and 
multiply by C. Then what should a and { be? 


While it is generally easier to use the first form with A and B to solve for the initial 
conditions, the second form is much more natural to use for interpretation of physical systems, 
since the constants C’ and 6 have nice physical interpretation. Write the solution as 


x(t) = Ccos(wot — 4). 


This is a pure-frequency oscillation (a sine wave). The amplitude is C, wo is the (angular) 
frequency, and 6 is the so-called phase shift. The phase shift just shifts the graph left or 
right. We call wo the natural (angular) frequency. This entire setup is called simple harmonic 
motion. 
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Let us pause to explain the word angular before the word frequency. The units of wo are 
radians per unit time, not cycles per unit time as is the usual measure of frequency. Because 
one cycle is 27 radians, the usual frequency is given by 5°. It is simply a matter of where we 
put the constant 27, and that is a matter of taste. 

The period of the motion is one over the frequency (in cycles per unit time) and hence a 
That is the amount of time it takes to complete one full cycle. 


Example 2.4.1: Suppose that m = 2kg and k = 8N/m. The whole mass and spring setup is 
sitting on a truck that was traveling at 1™/s. The truck crashes and hence stops. The mass 
was held in place 0.5 meters forward from the rest position. During the crash the mass gets 
loose. That is, the mass is now moving forward at 1™/s, while the other end of the spring is 
held in place. The mass therefore starts oscillating. What is the frequency of the resulting 
oscillation? What is the amplitude? The units are the mks units (meters-kilograms-seconds). 


Solution: The setup means that the mass was at half a meter in the positive direction 
during the crash and relative to the wall the spring is mounted to, the mass was moving 
forward (in the positive direction) at 1™/s. This gives us the initial conditions. 

So the equation with initial conditions is 


2x" + 8x = 0, 2(0) = 0.5, # (0) =, 


We directly compute wo = J] = /4= 
2. Hence the angular frequency is 2. The 
usual frequency in Hertz (cycles per second) | 00 
is 2/an = Yn 0.318. 

The general solution is 


x(t) = Acos(2t) + Bsin(2t). 


Letting x(0) = 0.5 means A = 0.5. Then 
a'(t) = —2(0.5) sin(2t) + 2B cos(2t). Letting Figure 2.4: Simple undamped oscillation. 
x'(0) = 1 we get B = 0.5. Therefore, the 

amplitude is C = VA? + B? = 0.25 + 0.25 = V0.5 = 0.707. The solution is 


z(t) = 0.5 cos(2t) + 0.5 sin(2¢). 
A plot of x(t) is shown in Figure 2.4. | 
In general, for free undamped motion, a solution of the form 
x(t) = Acos(wot) + Bsin(wot), 


corresponds to the initial conditions (0) = A and 2/(0) = woB. Therefore, it is easy to figure 
out A and B from the initial conditions. The amplitude and the phase shift can then be 
computed from A and B. In the example, we have already found the amplitude C’. Let us 
compute the phase shift. We know that tané = ®/a = 1. We take the arctangent of 1 and 
get 7/4 or approximately 0.785. We still need to check if this 6 is in the correct quadrant 
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(and add x to 6 if it is not). Since both A and B are positive, then 6 should be in the first 
quadrant, */4 radians is in the first quadrant, so 6 = 7/4. 

Note: Many calculators and computer software have not only the atan function for 
arctangent, but also what is sometimes called atan2. This function takes two arguments, B 
and A, and returns a 0 in the correct quadrant for you. 


2.4.2 Free damped motion 


Let us now focus on damped motion. Let us rewrite the equation 
mz" + 2'+kxr =0, 


as 
x” + 2px’ +urr = 0, 


lk 
Wo = ey ae 
m 2m 


r? + 2pr +p =0. 


where 


The characteristic equation is 


Using the quadratic formula we get that the roots are 


r=—pty/p?—«u. 


The form of the solution depends on whether we get complex or real roots. We get real roots 
if and only if the following number is nonnegative: 


2 
2 2 1) 5 yam 
pe &o . m 4m? 


The sign of p” — w? is the same as the sign of y? — 4km. Thus we get real roots if and only if 
7? — 4km is nonnegative, or in other words if y? > 4km. If these look familiar, that is not 
surprising, as they are the same as the conditions we had for the different types of roots in 
second order constant coefficient equations. 


Overdamping 


When 7” — 4km > 0, the system is overdamped. In this case, there are two distinct real roots 
r; and rg. Both roots are negative: As \/p? — wé is always less than p, then —p + \/p? — ue 
is negative in either case. 
The solution is 
a(t) = Cye™ + Coe™. 


Since r1,7r2 are negative, z(t) + 0 as t oo. Thus the mass will tend towards the rest 
position as time goes to infinity. For a few sample plots for different initial conditions, see 
Figure 2.5 on the following page. 
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No oscillation happens. In fact, the graph 
crosses the x-axis at most once. To see why, 
we try to solve 0 = Cye™'+ Cye"". Therefore, 


Cye™! = —Cye™! and using laws of exponents 
we obtain 10 10 
I 

—C; = gerne 

Cy 05 05 
This equation has at most one solution t > 0. 
For some initial conditions the graph never 4. aS 
crosses the x-axis, as is evident from the 
sample graphs. 0 28 30 75 700 


Example 2.4.2: Suppose the mass is re- Figure 2.5: Overdamped motion for several 
leased from rest. That is 2(0) = x and | different initial conditions. 
x'(0) = 0. Then 

xO 


t = rat rit . 
x(t) aes (rie roe") 


It is not hard to see that this satisfies the initial conditions. 


Critical damping 


When y? — 4km = 0, the system is critically damped. In this case, there is one root of 
multiplicity 2 and this root is —p. Our solution is 


z(t) = Cie + Cote™. 


The behavior of a critically damped sys- 
tem is very similar to an overdamped system. : 
After all a critically damped system is in i 
some sense a limit of overdamped systems. 
Even though our models are only approxima- 
tions of the real world problem, the idea of 


critical damping can be helpful in optimizing 2 
systems. Figure 2.6 shows how the solution 
to 
" / 
x + YX +z=0 ° 2 4 6 8 10 12 14 16 18 


for different values of y and initial conditions 7 
x«(0) = 4 and 2’'(0) = 0. This solution is 
critically damped if 7 = 2, as that will give |p igure 2.6: Overdamped and critically damped 
us a repeated root in the characteristic equa- | motion for a! +a! +2 =0 for y =2,4,8. 
tion. Comparing these solutions, we see that 

the critically damped solution gets back to 

equilibrium faster than any of the more overdamped solution. When trying to design a 
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system, if we want it to settle back to the zero point as quickly as possible, then we should 
try to get as closed to critically damped as possible. Even though we are always a little 
bit underdamped or a little bit overdamped, getting as close as possible will give the best 
possible result for returning to equilibrium. 


Underdamping 


When 77 — 4km < 0, the system is under- 
damped. In this case, the roots are complex. 


—p = \/ a "ae we 0.5 0.5 


0 El 
1.0 f 1.0 


/ 
= —pt+ V-14/u2 — p? 
. 0.0 ay, <——— -_ 
= —pt iw, qs a 
where w, = \/w? — p?. Our solution is 0s #8 


a(t) =e (Acos(wit) + Bsin(wt)), 


OE Figure 2.7: Underdamped motion with the en- 


a(t) = Ce cos(w)t — 6). velope curves shown. 
An example plot is given in Figure 2.7. Note 
that we still have that x(t) > 0 as t > oo. 

The figure also shows the envelope curves Ce~' and —Ce~”’. The solution is the oscillating 
line between the two envelope curves. The envelope curves give the maximum amplitude of 
the oscillation at any given point in time. For example, if you are bungee jumping, you are 
really interested in computing the envelope curve as not to hit the concrete with your head. 

The phase shift 6 shifts the oscillation left or right, but within the envelope curves (the 
envelope curves do not change if 6 changes). 

Notice that the angular pseudo-frequency* or quasi-frequency becomes smaller when the 
damping 7 (and hence p) becomes larger. This makes sense. When we change the damping 
just a little bit, we do not expect the behavior of the solution to change dramatically. If we 
keep making y larger, then at some point the solution should start looking like the solution 
for critical damping or overdamping, where no oscillation happens. So if y? approaches 4km, 
we want w; to approach 0. Since w; = \/w9 — p? with p = = and uy = Vkm, we have that 


which does go to zero as 7” gets closer to 4mk. 

On the other hand, when ¥ gets smaller, w; approaches wo (w; is always smaller than wp), 
and the solution looks more and more like the steady periodic motion of the undamped case. 
The envelope curves become flatter and flatter as y (and hence p) goes to 0. 


*We do not call w; a frequency since the solution x(t) is not really a periodic function. 
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2.4.3 Exercises 


Exercise 2.4.2: Consider a mass and spring system with a mass m = 2, spring constant 
k; = 3, and damping constant y = 1. 


a) Set up and find the general solution of the system. 

b) Is the system underdamped, overdamped or critically damped? 

c) If the system is not critically damped, find a y that makes the system critically damped. 
Exercise 2.4.3: Do Exercise 2.4.2 form = 3, k = 12, and y = 12. 


Exercise 2.4.4: Using the mks units (meters-kilograms-seconds), suppose you have a spring 
with spring constant 4N/m. You want to use it to weigh items. Assume no friction. You place 
the mass on the spring and put it in motion. 


a) You count and find that the frequency is 0.8 Hz (cycles per second). What is the mass? 
b) Find a formula for the mass m given the frequency w in Hz. 


Exercise 2.4.5:* A mass of 2 kilograms is on a spring with spring constant k newtons per 
meter with no damping. Suppose the system is at rest and at time t = 0 the mass is kicked 
and starts traveling at 2 meters per second. How large does k have to be to so that the mass 
does not go further than 3 meters from the rest position? 


Exercise 2.4.6: Suppose we add possible friction to Exercise 2.4.4. Further, suppose you do 
not know the spring constant, but you have two reference weights 1kg and 2kg to calibrate 
your setup. You put each in motion on your spring and measure the quasi-frequency. For the 
1kg weight you measured 1.1 Hz, for the 2kg weight you measured 0.8 Hz. 


a) Find k (spring constant) and y (damping constant). 


b) Find a formula for the mass in terms of the frequency in Hz. Note that there may be 
more than one possible mass for a given frequency. 


c) For an unknown object you measured 0.2 Hz, what is the mass of the object? Suppose 
that you know that the mass of the unknown object is more than a kilogram. 


Exercise 2.4.7: Suppose you wish to measure the friction a mass of 0.1 kg experiences as it 
slides along a floor (you wish to find y). You have a spring with spring constant k = 5 N/m. 
You take the spring, you attach it to the mass and fix it to a wall. Then you pull on the 
spring and let the mass go. You find that the mass oscillates with quasi-frequency 1 Hz. 
What is the friction? 


Exercise 2.4.8:* A 5000kg railcar hits a bumper (a spring) at 1™/s, and the spring 
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compresses by 0.1m. Assume no damping. 
a) Find k. 


b) How far does the spring compress when a 10000 kg railcar hits the spring at the same 
speed? 


c) If the spring would break if it compresses further than 0.3m, what is the maximum 
mass of a railcar that can hit it at 1™/s? 


d) What is the maximum mass of a railcar that can hit the spring without breaking at 
2 m/s? 


Exercise 2.4.9: When attached to a spring, a 2kg mass stretches the spring by 0.49 m. 
a) What is the spring constant of this spring? Use 9.8™/s? as the gravity constant. 


b) This mass is allowed to come to rest, lifted up by 0.4m and then released. If there is 
no damping, set up and solve an initial value problem for the position of the mass as a 
function of time. 


c) For a next experiment, you attach a dampener of coefficient 16Ns/m to the system, 
and give the same initial condition. Set up and solve an initial value problem for the 
position of the mass. What type of “dampening” would be used to characterize this 
situation? 


Exercise 2.4.10:* A mass of m kg is on a spring with k = 3N/m and c = 2Ns/m. Find the 
mass mo for which there is critical damping. If m < mo, does the system oscillate or not, 
that is, is it underdamped or overdamped? 


Exercise 2.4.11:* Suppose we have an RLC circuit with a resistor of 100 milliohms (0.1 
ohms), inductor of inductance of 50 millihenries (0.05 henries), and a capacitor of 5 farads, 
with constant voltage. 


a) Set up the ODE equation for the current I. 
b) Find the general solution. 
c) Solve for I(0) = 10 and I'(0) = 0. 


Exercise 2.4.12: For RLC circuits, we can use either charge or current to set up the 
equation. Let’s see how the two of those compare. 


a) Assume that we have an RLC circuit with a 30 millihenry inductor, a 120 milliohm 
resistor, and a capacitor with capacitance 29/3 F. Set up a differential equation for the 
charge on the capacitor as a function of time. 


b) Use the same circuit to set up a differential equation for the current through the circuit 
as a function of time. How do these equations relate? 
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c) Find the general solution to each of these equations. 
d) Solve the initial value problem for the charge with Q(0) = 1/2C and Q’(0) = 0. 


e) Using the fact that I = Q’, determine the appropriate initial conditions needed for I in 
order to solve for the current in this same setup (with those initial values for charge). 


f) Now, we'll do the same in the other direction. Solve the initial value problem for current 
with I(0) = 2A and I'(0) = 14/s, and see what the initial conditions would be for Q(t) 
for this setup. 


Exercise 2.4.13: Assume that the system my” + yy’ + ky = 0 is either critically or 
overdamped. Prove that the solution can pass through zero at most once, regardless of initial 
conditions. Hint: Try to find all values of t for which y(t) = 0, given the form of the solution. 
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2.5 Nonhomogeneous equations 


Attribution: [JL], §2.6. 


After this section, you will be able to: 


Find the corresponding homogeneous equation for a non-homogeneous equation, 
Use the method of undetermined coefficients to solve non-homogeneous equations, 
Use variation of parameters to solve non-homogeneous equations, and 

Solve for the necessary coefficients to solve initial value problems for non- 
homogeneous equations. 


2.5.1 Solving nonhomogeneous equations 


We have solved linear constant coefficient homogeneous equations. What about nonhomoge- 
neous linear ODEs? For example, the equations for forced mechanical vibrations, where we 
add a “forcing” term, which is a function on the right-hand side of the equation. That is, 
suppose we have an equation such as 


y’ +5y' + 6y = 2r +1. (2.6) 


We will write L[y] = 2x +1, where Ly] represents the entire left-hand side of y” + 5y’ + 6y, 
when the exact form of the operator is not important. We solve (2.6) in the following manner. 
First, we find the general solution y,. to the associated homogeneous equation 


y’ + 5y' + 6y = 0. (2.7) 


We call y, the complementary solution. Next, we find a single particular solution yp to (2.6) 
in some way (that is the point of this section). Then 


Y = Ve t+ Up 


is the general solution to (2.6). We have Lly.] = 0 and L{y,| = 20+ 1. As L is a linear 
operator we verify that y is a solution, L[y] = L[ye+ yp] = Lye] + L[yp] = 0+ (2 + 1). Let 
us see why we obtain the general solution. 

Let y, and y, be two different particular solutions to (2.6). Write the difference as 
W = Yp — Yp. Then plug w into the left-hand side of the equation to get 


w" + 5w! + 6w = (yy + Sy, + 6yp) — (> + 5g, + GY) = (22 + 1) — (Qe +1) = 0. 


Using the operator notation the calculation becomes simpler. As L is a linear operator we 
write 
L[w| = L[yp — Hp] = Llyp] — LlGp] = (22 + 1) — (22 + 1) = 0. 
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So W = Yp — Yp is a solution to (2.7), that is Lw = 0. However, we know what all solutions 
to Lw = 0 look like, as this is a homogeneous equation that we have solved previously. 
Therefore, any two solutions of (2.6) differ by a solution to the homogeneous equation (2.7). 
The solution y = y. + Yp includes all solutions to (2.6), since y, is the general solution to the 
associated homogeneous equation. 


Theorem 2.5.1 


Let Lly] = f(a) be a linear ODE (not necessarily constant coefficient). Let y. be the 
complementary solution (the general solution to the associated homogeneous equation 


Lly| = 0) and let yp, be any particular solution to Lly] = f(x). Then the general 
solution to Lily] = f(x) is 


Y = Yo Up- 


The moral of the story is that we can find the particular solution in any old way. If we 
find a different particular solution (by a different method, or simply by guessing), then we 
still get the same general solution. The formula may look different, and the constants we 
have to choose to satisfy the initial conditions may be different, but it is the same solution. 


2.5.2 Undetermined coefficients 


The trick is to somehow, in a smart way, guess one particular solution to (2.6). Note that 
2x +1 is a polynomial, and the left-hand side of the equation (with all of the derivatives) 
will still be a polynomial if we let y be a polynomial of the same degree. Let us try 


GC; = A+B. 
We plug y, into the left hand side to obtain 
yp + Sy), + 6yp = (Ax + B)” + 5(Ax + B)’ + 6(Ax + B) 
=04+5A+6Ar4+ 6B = 6Azr + (544+ 6B). 


So 6Axz + (54+ 6B) = 2x41. If we match up the coefficients of x in this equation, we get 
that 6A = 2 or A = 1/3. In order for the constant terms to match, we need that 54+6B = 1. 
Since we know the value of A, this tells us that B = -1/o. That means y, = $2 — 5 = 2+. 
Solving the complementary problem (exercise!) we get 


C= Ce" Ge. 
Hence the general solution to (2.6) is 


3x2 —1 


y = Cie + Cge*" + 
Now suppose we are further given some initial conditions. For example, y(0) = 0 and 
y'(0) = 1/3. First find y’ = —2C,e~** — 3Cze~** + 1/3. Then 


1 1 1 
0=y7(0) =C, Ca— 5) = y'(0) = —201 — 3C2 + 5. 
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We solve to get C; = !/3 and C, = ~2/9. The particular solution we want is 


1 5, 2 3, 30-1 3e7** — 2e~8* + 3x —-1 
=e e = 
3 9 9 9 


Exercise 2.5.1: Check that y really solves the equation (2.6) and the given initial conditions. 


Note: A common mistake is to solve for constants using the initial conditions with y, and 
only add the particular solution y, after that. That will not work. You need to first compute 
Y = Ye + Yp and only then solve for the constants using the initial conditions. 


A right-hand side consisting of exponentials, sines, and cosines can be handled similarly. 


Example 2.5.1: One example of this is 
y” + 2y' + 2y = cos(2z). 


Solution: Let us find some y,. We start by guessing that the solution includes some multiple 
of cos(2a). We try 
Yp = Acos(2z). 


Plugging this into the differential equation gives 


—4A cos(2x) +2 (—2A sin(2x)) +2 (Acos(2x)) = cos(2z). 
-_—— a s, Se 


/ 


Yp Yp Yp 
Simplifying this expression gives 
—2A cos(2x) — 4Asin(2x) = cos(2z) 


and we have a problem. Since there is no sine term on the right-hand side, we are forced to 
pick A = 0, which means our non-homogeneous solution is zero, and that’s not good. What 
happened here? In the previous example, when we differentiated a polynomial (as part of the 
Yp guess) the function stayed a polynomial, and so we did not add any new types of terms. 
In this case, however, when we differentiate the cosine term in our guess, it becomes a sine, 
which we did not have in our initial guess. 

Thus, we will also want to add a multiple of sin(2x) to our guess since derivatives of 
cosine are sines. We try 

Yp = Acos(2x) + Bsin(2z). 


We plug yp into the equation and we get 


—4A cos(2x) — 4B sin(2x) +2 (—2Asin(2x) + 2B cos(2z)) 
=Vee_aQCoa=:::-l_” a 


/ 


Up Up 


+ 2(Acos(2x) + 2B sin(2x)) = cos(2z), 
eS 
Yp 
or 
(-4A + 4B + 2A) cos(2x) + (-4B — 4A + 2B) sin(2x2) = cos(2z). 
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The left-hand side must equal to right-hand side. Namely, —44 + 4B+ 2A = 1 and 
—-4B —4A+2B=0. So -24+4B=1 and 24+ B=0. We can solve this system of 
equations to get that A = —1/10 and B = 1/5. So 


— cos(2z) + 2 sin(2z) 
10 


Yp = Acos(2x) + Bsin(2x) = 


Similarly, if the right-hand side contains exponentials we try exponentials. If 
Ly] = Be 


we try y = Ae®” as our guess and try to solve for A. 


When the right-hand side is a multiple of sines, cosines, exponentials, and polynomials, 
we can use the product rule for differentiation to come up with a guess. We need to guess 
a form for y, such that L[y,| is of the same form, and has all the terms needed to for the 
right-hand side. For example, 


Ly] = (1 + 32?) e~* cos(r2). 
For this equation, we guess 
Yp = (A+ Bu + Cx?) e~* cos(rxz) + (D+ Ex + Fx’) e™* sin(r2). 


We plug in and then hopefully get equations that we can solve for A, B, C, D, E, and F. As 
you can see this can make for a very long and tedious calculation very quickly. C’est la vie! 


There is one hiccup in all this. It could be that our guess actually solves the associated 
homogeneous equation. That is, suppose we have 


y” -_ oy = ee, 


We would love to guess y = Ae**, but if we plug this into the left-hand side of the equation 
we get 
y” — 9y = 9Ae** — 9Ae®* = 0 # €**. 


There is no way we can choose A to make the left-hand side be e®*. The trick in this case is 
to multiply our guess by x to get rid of duplication with the complementary solution. That 
is first we compute y, (solution to L[y] = 0) 


Yo = Cre ** + Coe**, 
and we note that the e®* term is a duplicate with our desired guess. We modify our guess 
to y = Axe** so that there is no duplication anymore. Let us try: y’ = Ae®* + 3Are** and 


y" = 6Ae** + 9Aze**, so 


y” — 9y = 6Ae*™* + 9Are®” — 9Are*™ = 6 Ae. 
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Thus 6Ae** is supposed to equal e?”. Hence, 64 = 1 and so A = 1/6. We can now write the 
general solution as 


1 
Y= Vo+ Yp = Cre + Coe + = zee. 


Notice that the term of the form ze®* does not show up on the left-hand side after 
differentiating the equation, and the only term that survives is the e?” term that showed up 
from the derivatives. This works out because e®” solves the homogeneous problem. With that 
though, make sure to remember to include the xe** when you write out the general solution 
at the end of the problem, because it does appear there. 


It is possible that multiplying by x does not get rid of all duplication. For example, 
y” _ Gy dt Oy = ee", 


The complementary solution is y. = C,e** + Cove”. Guessing y = Axe®” would not get us 
anywhere. In this case we want to guess y, = Axe". Basically, we want to multiply our 
guess by x until all duplication is gone. But no more! Multiplying too many times will not 
work (in that case, the derivatives won’t actually get down to the plain e** term that you 
need in order to solve the problem). 


Finally, what if the right-hand side has several terms, such as 
Lly] = e?* + cos 2. 


In this case we find u that solves L[u] = e?” and v that solves L{v] = cosx (that is, do each 
term separately). Then note that if y= u-+v, then L[{y] = e?” +cosx. This is because L is 
linear; we have Liy] = L{u + v] = L[u] + L[v] = e?” + cos z. 

To summarize all of this, we can make a table of the different guesses we should make 
given the form of the right hand side. 


Right hand side Guess 
G0" + Op it” Eee Bae ag | Act Bette ENP 
et Ae 
cos ax Acosax+ Bsinax 
sin ax Acosax+ Bsinax 


e If there is a product of above terms, guess the product of the guesses. So, for a right 
hand side of xe*”, the guess should be (Ax + B)e*", and for a right hand side of x cos az, 
the guess should be (Az + B) cosax + (Ca + D)sinaz. 


e If any part solves the homogeneous problem, multiply that entire component by x until 
nothing does. 


Example 2.5.2: Find the solution to the initial value problem 


y” — 3y' — 4y = 2e* + Asin(x) y(0) = —2, y'(0)=1 


152 CHAPTER 2. HIGHER ORDER LINEAR ODES 


Solution: To start this problem, we look for the solution to the homogeneous problem. The 
characteristic equation for the left hand side is r? — 3r — 4, which factors as (r — 4)(r + 1). 
Therefore the general solution to the homogeneous problem (or the complementary solution) 
is 

ye(x) = Cre + Coe. 

Next, we want to use undetermined coefficients to solve the non-homogeneous problem. 
Note that we have to wait until after this part to meet the initial conditions. Since our 
right-hand side is 2e~* + 4sin(a), we need to guess two components for the two different 
terms in this function. For the first term, we would want to guess Ae~*, but this function 
solves the homogeneous problem. Therefore, we need to multiply by x to use Axe~” as our 
guess. For the sine term, we need to guess both sine and cosine, so we add B sin(x) + C'cos(x) 
to our guess. Therefore, our total guess for the non-homogeneous solution is 


Yp(%) = Axe™* + Bsin(z) + C cos(z). 
We take two derivatives of this function and then plug it into the differential equation 
yp(x) = Axe~* + Bsin(x) + C cos(z) 
y) (x) = Ae” — Are~* + Bcos(x) — Csin(2) 
y), (a) = Axe~* — 2Ae~™* — Bsin(x) — C cos(x) 
so that 
y, — 3y,, — yp = (Are — 2Ae~* — Bsin(x) — C'cos(z)) 
— 3(Ae* — Axe” + Bcos(x) — C'sin(z)) 
— 4(Ave~* + Bsin(x) + Ccos(x)) 
which can be simplified to 
y, — 8y), — 4p = —5Ae~™ + (3B + 3C) sin(x) + (3C — 3B) cos(x). 


Since we want this to equal 2e~* + 4sin(x), this means that we need —5A = 2, so A = —2/s, 
as well as 3B + 3C = 4 and 3C' — 3B =0. The second of these implies that B = C’, and the 
first equation then gives that 6B = 4 or B = C = 2/3. Therefore, the general solution to this 
non-homogeneous problem is 


2 2 2 
y(x) = Cie” + Cye~* — gre © + 3 sin(x) + 3 cos(z). 


Now we can look to meet the initial conditions. We want to differentiate this expression 
to get 


2 2 2 2 
f(z) =4C\e* = Core™ = Be pre” 3 cos(a”) — 3 sin(z) 
and then plug zero into both y and 7 to get that 
2 
y(0) tC ga =-—2 


2D 
(OV=4C pCa Seb Sa 
y (0) 1 2 573 
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which gives rise to the system 


8 11 
Ci +C,=-= 4C, — Cp = —. 
1+ Og 3 1 2= 75 
Adding the equations together gives 5C, = -2 so that C, = -2 and then Cy = —%. 
Therefore the solution to the initial value problem is 
29 171 2 2 2 
un) = “eee — wae — gre * + 3 sin(x) + a cos(2). 


Exercise 2.5.2: Verify that this y(x) solves the initial value problem! 


2.5.3 Variation of parameters 


The method of undetermined coefficients works for many basic problems that crop up. But it 
does not work all the time. It only works when the right-hand side of the equation L{y] = f(x) 
has finitely many linearly independent derivatives, so that we can write a guess that consists 
of them all. Some equations are a bit tougher. Consider 


y’ +y=tanz. 


Each new derivative of tanxz looks completely different and cannot be written as a linear 
combination of the previous derivatives. If we start differentiating tan x, we get: 


sec?a, 2sec?x tanz, 4sec*x tan? x +2sec'z, 


8sec? 2 tan?x+16sectx tanz, 16sec*?x tantx + 88sec’ x tan? x + 16sec® x, 


This equation calls for a different method. We present the method of variation of 
parameters, which handles any equation of the form Lily] = f(x), provided we can solve 
certain integrals. For simplicity, we restrict ourselves to second order constant coefficient 
equations, but the method works for higher order equations just as well (the computations 
become more tedious). The method also works for equations with nonconstant coefficients, 
provided we can solve the associated homogeneous equation. 

Perhaps it is best to explain this method by example. Let us try to solve the equation 


Ly} = y" +y = tanz. 


First we find the complementary solution (solution to Liy.] = 0). We get y. = Cry1 + Coye, 
where y; = cosx and yo = sinz. To find a particular solution to the nonhomogeneous 
equation we try 

Up = Y¥ = U1Y1 + U2Y2, 
where uw; and wz are functions and not constants. We are trying to satisfy Lly] = tanz. That 
gives us one condition on the functions wu; and ug. Compute (note the product rule!) 


y! = (uyyr + ugya) + (ury + u2y). 
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We can still impose one more condition at our discretion to simplify computations (we 
have two unknown functions, so we should be allowed two conditions). We require that 
(wiyi + usy2) = 0. This makes computing the second derivative easier. 


/ / / 
Y = UY, + U2Yo; 
y" = (uy, + ugys) + (uy + uayg). 


Since y; and y2 are solutions to y” + y = 0, we find y// = —y; and y = —yp. (If the equation 
was a more general y” + p(x)y’ + q(x)y = 0, we would have y/ = —p(x)y; — q(x)y;.) So 


y" = (uy, + ugys) — (urys + uae). 
We have (uiyi + u2y2) = y and so 
y" = (uy) + ugyd) — y, 


and hence 
y" +y = Ly] = uy, + udyd. 


For y to satisfy Lly] = f(a) we must have f(x) = uly, + ugys. 
What we need to solve are the two equations (conditions) we imposed on wu; and ug: 


We solve for wu‘, and uj) in terms of f(x), y; and yo. We always get these formulas for any 
Lly] = f(x), where L[y] = y” + p(ax)y’ + q(a)y. There is a general formula for the solution we 
could just plug into, but instead of memorizing that, it is better, and easier, to just repeat 
what we do below. In our case the two equations are 


u, cos(x) + uy sin(x) = 0, 


—u' sin(x) + us cos(x) = tan(z). 
Hence 


u', cos(x) sin(x) + ut, sin?(x) = 0, 


—u} sin(x) cos(x) + us cos*(x) = tan(zx) cos(x) = sin(z). 
And thus 


u)(sin?(x) + cos*(x)) = sin(2), 
ti = Sinn); 
— sin*(x) 


uy = ae. = —tan(z) sin(z). 
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We integrate ui, and us to get wu, and uy. 


uu = fu az = [tan sin(x) dr = 5 In 
U2 = fe dx = [sx dx = —cos(x). 


So our particular solution is 


1 i -1 
Yp = U1Y1 + U2y2 = 5 cos(z) In nee + cos(x) sin(x) — cos(zx) sin(a) = 
-—] 
= —cos(x) In sly, 
sin(x) + 1 
The general solution to y” + y = tan x is, therefore, 
1 i -1 
y = Cicos(x) + Cg sin(x) + 5 cos(z) In me a | 


In more generality, we can take the system of equations 


/ / 
U1Y1 + Ugy2 = 0, 
uy, + usys = f(a). 


and solve out for u, and uw) using elimination. If we do that, we get that 


yo(x) f(x) yi(x) f(x) 


yi(x)ys(x) — yi (2) y2(z) yi (x) yo(x) — yi (2) yo(x) 


We know that solving the equations this way will work out because we start with the 
assumption that y; and ye are linearly independent solutions, and the denominator of both 
of these fractions is exactly what we know is not zero from this assumption. Therefore, both 
of these functions can be written this way, we can integrate both of them, and set up our 
particular solution of the form y,(#) = uyy1 + uzye to get 


ye cactey [tke ctey [led ; 
vole) = In if uae if AeuO=vene 


where zo is any conveniently chosen value (usually zero). Notice the use of r as a dummy 
variable here to separate the functions being integrated from the actual variable that shows 
up in the solution. This formula will always work for finding a particular solution to a 
non-homogeneous equation given that we know the solution to the homogeneous equation, 
but we may not be able to work out the integrals explicitly. This is the downside of this 
method, it may always work, but can be very tedious and may not result in nice, closed-form 
expressions like we might get from other methods. 
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Example 2.5.3: Find the general solution to the differential equation 
y” + 4y' + 3y = 2 +2 
using both undetermined coefficients and variation of parameters. 


Solution: For both methods of solving non-homogeneous equations, we need the solution to 
the homogeneous problem. For this equation, the characteristic polynomial is r? + 4r + 3, 
which factors as (r + 1)(r + 3), so the general solution to the homogeneous problem is 


ye(z) = Cye* + Coe **. 


To use undetermined coefficients, we need to get the appropriate guess for the right-hand 
side, which in this case is y,(x) = Ae®**+ B. Plugging this in to the differential equation gives 


9Ae** + 4(3.Ae**) + 3(Ae** + B) = e** +2 


which simplifies to 
24Ae* + 3B = e** +2 
so that A = 1/24 and B = 2/3. Thus, the general solution to the non-homogeneous equation is 
—2 Se job apy 4 
y(x) = Cye™* + Coe + Ta 


In order to use variation of parameters, we let y;(2) = e~* and yo(x) = e~** be the two 
linearly independent solutions that we found to the homogeneous problem. Our right-hand 
side function is f(x) = e®” + 2 and we can compute the expression 


yi (x)y5(x) — yy (z)yo(z) = e7*(—3e7**) — (—e~*)e**” = —2e~™. 


Therefore, we can use the formulas from the method of variation of parameters to compute 
that 


i Se yo(x) f(x) 2 ee) 1 ae - 
yi(x)ya(2) — yi (x) y2(x) =de-* 2 
aes n(x) f(z) = Ere) a _1 cee _ @ 
yi (x)ya(2) — yi (x) yo(x) as 2 
Then we can compute 
U, = se +e7+C, tis = — ae — “ 1 Cp. 


Then, we can write out the full general solution as y(x) = uy(x)y (x) + U2(x)yo(x) or 


1 1 1 
yes (se + e* 4 C1) Leo (-5¢"- a0 Ca) 


1 1 1 
= ae +14 Cie* — all mie + Coe ** 


which, after combining the terms, is the same as the solution that we obtained via undeter- 
mined coefficients. _| 
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2.5.4 Exercises 

Exercise 2.5.3: Find a particular solution of y” — y' — 6y = e*. 
Exercise 2.5.4: Find a particular solution of y” — 4y'! + 4y = e?”. 
Exercise 2.5.5:* Find a particular solution to y" — y' + y = 2sin(32) 


Exercise 2.5.6: Solve the initial value problem y” + 9y = cos(3z) + sin(3x) for y(0) = 2, 
y'(0) = 1. 


Exercise 2.5.7: Set up the form of the particular solution but do not solve for the coefficients 
for y) _ Qy!" i yl!" = er. 


Exercise 2.5.8: Set up the form of the particular solution but do not solve for the coefficients 
for yY — Qy"+y" =e" +e+4+sinz. 


Exercise 2.5.9:* Solve y" + 2y' +y = x”, y(0) =1, y’(0) = 2. 


Exercise 2.5.10: Use the method of undetermined coefficients to solve the DE y" + 4y' = 
2t + 30. 


Exercise 2.5.11: 
a) Using variation of parameters find a particular solution of y" — 2y' + y = e”. 
b) Find a particular solution using undetermined coefficients. 
c) Are the two solutions you found the same? See also Exercise 2.5.27. 
Exercise 2.5.12:* 
a) Find a particular solution to y” + 2y = e® + 2°. 
b) Find the general solution. 
Exercise 2.5.13: Find the general solution to y” — 3y' — 4y = e** +1. 
Exercise 2.5.14: Find the general solution to y"” — 2y' — 5y = sin(3t) + 2 cos(3t). 
Exercise 2.5.15: Find the general solution to y" — 4y' — 2ly =e"! +e”. 
Exercise 2.5.16: Find the general solution to y — 2y’ +y =e! —-t. 
Exercise 2.5.17: Find the general solution to y"”+4y = sec(2t) using variation of parameters. 


Exercise 2.5.18: Find the solution of the initial value problem y" — 2y' — 15y = e* +3, 
y(0) = 2, (0) = —1 


Exercise 2.5.19: Find the solution of the initial value problem y” + 4y' + 5y = cos(3t) +t, 
y(0) = 0, y'(0) = 2. 
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Exercise 2.5.20: The following differential equations are all related. Find the general 
solution to each of them and compare and contrast the different solutions and the methods 
used to approach them. 


a) y” — 2y' — 15y = e' + 5e~* 
b) y” — 2y' — 15y = 2e7“ + 3e~* 
c) y” — 2y’ — 15y = 3cos(2t) 

d) y” — 2y' — 15y = 2e* — sin(t) 


Exercise 2.5.21: The following differential equations are all related. Find the general 
solution to each of them and compare and contrast the different solutions and the methods 
used to approach them. 


a) y” + 4y’ + 3y = e7* + 3e” 


b) yl" = 2Qy! ao 5y = e2t 2 3e%t 
c) y” — 3y’ — 10y = e* + 3e” 


d) y” — 8y' + 16y = ec + 3e” 


Exercise 2.5.22: Find a particular solution of y" — 2y’ + y = sin(x?). It is OK to leave the 
answer as a definite integral. 


Exercise 2.5.23: Use variation of parameters to find a particular solution of y" —y = ae 


Exercise 2.5.24: Recall that a homogeneous Euler equation is one of the form t?y” + aty’ + 
by = 0 and is solved by using the guess y(t) = t” and solving for the potential values of r. 


a) Solve t?y” — 2ty' — 10y = 0. 


b) Let y, and yg be a fundamental set for the above equation. Use the variation of 


yogt) in n= | yi g(t) 


; ~dt to solve the 


parameters equations uy = — / 7 
Y1Y2 — Y2yi 


YY — Y2Vh 
7 ae aes 


non-homogeneous equation y” — ronan 


(Do not attempt method of undetermined coefficients instead; it won’t work.) 


Exercise 2.5.25: For an arbitrary constant c find the general solution to y” —2y = sin(x+c). 


Exercise 2.5.26: For an arbitrary constant c find a particular solution to y” — y = e™. 
Hint: Make sure to handle every possible real c. 


Exercise 2.5.27: 
a) Using variation of parameters find a particular solution of y” — y = e”. 
b) Find a particular solution using undetermined coefficients. 


c) Are the two solutions you found the same? What is going on? 
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2.6 Forced oscillations and resonance 


Attribution: [JL], §2.6. 


After this section, you will be able to: 


e Write differential equations to model forced oscillators (like masses on springs), 


e Identify when beats, pure resonance, and practical resonance can occur, and 
e Use proper terminology around transient and steady periodic solutions when 
discussing these problems. 


Let us return back to the example of a mass on a spring. k F(t) 
We examine the case of forced oscillations, which we did not m -~ >» 
yet handle. That is, we consider the equation 


max" +a' + kx = F(t), damping c 


for some nonzero F(t). The setup is again: m is mass, ¥ is friction, k is the spring constant, 
and F'(t) is an external force acting on the mass. 

We are interested in periodic forcing, such as noncentered rotating parts, or perhaps loud 
sounds, or other sources of periodic force. 


2.6.1 Undamped forced motion and resonance 
First let us consider undamped (y = 0) motion. We have the equation 
ma” + kx = Fo cos(wt). 


This equation has the complementary solution (solution to the associated homogeneous 
equation) 
Le = C\ cos(wot) + Cz sin(wot), 


where Wp = J */m is the natural frequency (angular). It is the frequency at which the system 
“wants to oscillate” without external interference. 

Suppose that wo # w. We try the solution x, = Acos(wt) and solve for A. We do not 
need a sine in our trial solution as after plugging in we only have cosines. If you include a 
sine, it is fine; you will find that its coefficient is zero (I could not find a second rhyme). 

We solve using the method of undetermined coefficients. We find that 


Fo 
m(w — w?) 
We leave it as an exercise to do the algebra required. 

The general solution is 


i= cos(wt). 


x = C) cos(wot) + Co sin(wot) + 
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Written another way 


x = Ccos(wot — y) + cos(wt). 


0 
m/w — w*) 
The solution is a superposition of two cosine waves at different frequencies. 
Example 2.6.1: Take 

0.52" + 8a = 10cos(zt), #(0) = 0, (0) = 0. 


Solution: Let us compute. First we read off the parameters: w = 7, wo = 1/8/05 = 4, 
Fo = 10, m = 0.5. The general solution is 


x = C,cos(4t) + Cy sin(4t) + cos(7t). 


16 — 7? 


Solve for Cy and C> using the initial con- 
ditions: C = Zs and C; = 0. Hence 10 ® 


a (cos(mt) — cos(4t)). ; 5 


~ 16 — 72 


Notice the “beating” behavior in Fig- 
ure 2.8. First use the trigonometric identity 


A-B A+B 
2sin (=) sin ( ) =cosB-cosA 5 = 


10 -10 
0 5 10 15 20 


20 _ (4-7 . {4498 
L= (2sin (55+) sin ( 5 t)). Figure 2.8: Graph of 77%s (cos(mt) — cos(4t)). 


The function x is a high frequency wave mod- 
ulated by a low frequency wave. | 


The beating behavior can be experienced even more readily by considering a higher 
frequency and viewing the resulting function as a sound wave. A sound wave of frequency 
440 Hz produces and A4 sound, which is the A above middle C on a piano. This means that 
the function 

Lp(t) = sin(27 - 440t) 


will produce a sound wave equivalent to this A4 sound. In MATLAB, this can be done with 
the code 


omegaO = 440*2*pi; 
tVals = linspace(0, 5, 5*8192); 


testSound = sin(omega0*tVals) ; 
sound (testSound) ; 
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which will play this pitch for 5 seconds. Now, we want to see what happens if we take a 
mass-on-a-spring with this natural frequency and apply a forcing function with frequency 
close to this value. The following code assumes a forcing function of frequency 444 Hz. The 
multiple of wo in front of the forcing function is only for scaling purposes; otherwise the 
resulting sound would be too quiet. 


omega = 444*2*pi; 


syms ys(t); 

[V] = odeToVectorField(diff(ys, 2) + omega0“2*ys == omega0*cos(omega*t) ) ; 
MS = matlabFunction(V, 'vars', {'t', 'Y'}); 

soln = ode45(MS, [0,10], [0,0]); 


ySound = deval(soln, tVals) ; 
ySound = ySound(1, :); 
sound (ySound) ; 


A graph of the solution ySound can be found in Figure 2.9. This exhibits the beating 
behavior before on a large scale. The sound played during this code also shows the beating 
or amplitude modulation that can happen in these sorts of solutions. In terms of tuning 
instruments, these beats are some of the main things musicians will listen for to know if their 
instrument is close to the right pitch, but just slightly off. 


1 . 1 . ‘ . 
0 o8 1 #15 2 25 3 35 4 45 
«104 


Figure 2.9: Plot of ySound illustrating the beating behavior of interacting sound waves. 


Now suppose Wo = w. We cannot try the solution A cos(wt) and then use the method of 
undetermined coefficients, since we notice that cos(wt) solves the associated homogeneous 
equation. Therefore, we try x, = Atcos(wt) + Btsin(wt). This time we need the sine term, 
since the second derivative of tcos(wt) contains sines. We write the equation 


Ff 
x" +we = — cos(wt). 
m 
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Plugging x, into the left-hand side we get 


Fi 
2Bw cos(wt) — 2Aw sin(wt) = —2 cos(wt). 
m 


Hence A = 0 and B= ne Our particular solution is 
is 


fo 
2mw 


tsin(wt) and our general solution 


Fi 
x = Ci cos(wt) + C2 sin(wt) + ae tsin(wt). 


The important term is the last one (the particular solution we found). This term grows 
Fo 


without bound as t + oo. In fact it oscillates between yet and fot The first two terms 
only oscillate between +,/C? + CZ, which becomes smaller and smaller in proportion to the 
oscillations of the last term as ¢ gets larger. In Figure 2.10 we see the graph with C, = Cy = 0, 
i=2,i7= Lee 

By forcing the system in just the right 
frequency we produce very wild oscillations. 
This kind of behavior is called resonance or 
perhaps pure resonance. Sometimes reso- 
nance is desired. For example, remember 
when as a kid you could start swinging by 
just moving back and forth on the swing seat 
in the “correct frequency”? You were trying 
to achieve resonance. The force of each one 
of your moves was small, but after a while it 
produced large swings. 

On the other hand resonance can be de- 
structive. In an earthquake some buildings 
collapse while others may be relatively un- 
damaged. This is due to different buildings 
having different resonance frequencies. So figuring out the resonance frequency can be very 
important. 

A common (but wrong) example of destructive force of resonance is the Tacoma Narrows 
bridge failure. It turns out there was a different phenomenon at play*. 


Figure 2.10: Graph of +tsin(rt). 


2.6.2 Damped forced motion and practical resonance 


In real life things are not as simple as they were above. There is, of course, some damping. 
Our equation becomes 
ma" + a' + kx = Fy cos(wt), (2.9) 


for some y > 0. We solved the homogeneous problem before. We let 


ik 
p= Wo = mene 
2m m 


*K. Billah and R. Scanlan, Resonance, Tacoma Narrows Bridge Failure, and Undergraduate Physics 
Textbooks, American Journal of Physics, 59(2), 1991, 118-124, http://www.ketchum.org/billah/ 
Billah-Scanlan. pdf 
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We replace equation (2.9) with 
Fi 
aw" + Qpa! + wer = — cos(wt). 
m 


The roots of the characteristic equation of the associated homogeneous problem are 71,72 = 
—p+ /p? —we. The form of the general solution of the associated homogeneous equation 
depends on the sign of p* — wé, or equivalently on the sign of y? — 4km, as before: 


Cre™! + Cye™* if vy > 4km, 
Lo= < Cie + Cote if y? = 4km, 
e~* (Cy cos(w t) + Cysin(w;t)) if 7? < 4km, 


where w; = ,/w? — p?. In any case, we see that x,(t) > 0 as t > oo. 

Let us find a particular solution. There can be no conflicts when trying to solve for the 
undetermined coefficients by trying x, = Acos(wt) + Bsin(wt), because the solution to the 
homogeneous problem will always have exponential factors (since we have damping) and so 
there is no w where this will exactly match the form of the homogeneous solution. Let us 
plug in and solve for A and B. We get (the tedious details are left to reader) 


((wo — w?)B — QwpA) sin(wt) + ((w§ — w*)A + 2wpB) cos(wt) = “0 cos(wt). 


We solve for A and B: 
= (w§ — w?) Fo 
m(2uwp)* + m(u2 — w?)?’ 
2wpFo 


m(2up)* + mu? - Ww?) 


We also compute C = VA? + B? to be 


Fo 


my/ (2uop)? + (u8 — 2)? 


C= 


Thus our particular solution is 
(wo — w?) Fo 


> m(2wp)? + m(w? — w?) 


2wpFo 


m(2uwp)? + m(w2 — w?) 


5 cos(wt) + 5 sin(wt). 


Or in the alternative notation we have amplitude C and phase shift 6 where (if w 4 wo) 


B 2wp 
tan é = ri aoe 


Hence, 


Fo 


ma/ (2up)? + (w3 — w)? 


n= cos(wt — 0). 
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If w = wo, then A= 0, B=C = 5, and 6 = 7/2. 


2mwp? 


For reasons we will explain in a moment, we call x, the transient solution and denote it 
by x,. We call the x, from above the steady periodic solution and denote it by x,,. The 
general solution is 

L= Lo t+ Lp = Lip + Lop. 


The transient solution 7, = 2, goes to zero as t — oo, as all the terms involve an 
exponential with a negative exponent. So for large t, the effect of x4, is negligible and we 
see essentially only x,,. Hence the name transient. Notice that x,, involves no arbitrary 
constants, and the initial conditions only affect x;,. Thus, the effect of the initial conditions 
is negligible after some period of time. We might as well focus on the steady periodic solution 
and ignore the transient solution. See Figure 2.11 for a graph given several different initial 
conditions. 

The speed at which 2x; goes to zero de- 
pends on p (and hence y). The bigger p is 
(the bigger y is), the “faster” 24, becomes 
negligible. So the smaller the damping, the 
longer the “transient region.” This is consis- 
tent with the observation that when y = 0, 
the initial conditions affect the behavior for 
all time (i.e. an infinite “transient region” ). 


Let us describe what we mean by reso- 

nance when damping is present. Since there 
were no conflicts when solving with undeter- 
mined coefficient, there is no term that goes 
to infinity. We look instead at the maximum | Figure 2.11: Solutions with different initial 
value of the amplitude of the steady periodic | conditions for parameters k =1,m=1, Fo 
solution. Let C' be the amplitude of z,,. If 1,7=0.7, andw = 1.1. 
we plot C as a function of w (with all other 
parameters fixed), we can find its maximum. 
We call the w that achieves this maximum the practical resonance frequency. We call the 
maximal amplitude C(w) the practical resonance amplitude. Thus when damping is present 
we talk of practical resonance rather than pure resonance. A sample plot for three different 
values of y is given in Figure 2.12 on the next page. As you can see the practical resonance 
amplitude grows as damping gets smaller, and practical resonance can disappear altogether 
when damping is large. 

The main takeaways from Figure 2.12 on the facing page is that the amplitude can be 
larger than 1, which is the idea of resonance in this case. Based on Hooke’s law, we know 
that a constant force of magnitude Fo will stretch (or compress) a spring with constant k a 
length of Fo/k. If we take Fy = 1 and k = 1, as is done in Figure 2.12 on the next page, then 
the resulting magnitude should be 1. However, if we don’t use a constant force of magnitude 
Fo, but instead use an oscillatory force with frequency w of the form F(t) = Fo cos (wt), 
we get an amplitude of C(w). This graph indicates how the forcing frequency changes the 
amplitude of the resulting oscillation. Since the amplitude “should” be 1 based on Fo/k, if 
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Figure 2.12: Graph of C(w) showing practical resonance with parameters k =1, m=1, Fo = 1. 
The top line is with y = 0.4, the middle line with y = 0.8, and the bottom line with y = 1.6. 


C(w) > 1, then the frequency chosen is causing an increase in the amplitude, which is the 
idea of practical resonance. 

To find the maximum, or determine if there is a maximum, we need to find the derivative 
C’(w). Computation shows 


—2u (2p? + w* — w?) Fy 
3/2° 
m/((2up)? + (u2 — w2)?)” 


C"(w) = 


This is zero either when w = 0 or when 2p? + w? — wé = 0. In other words, C’(w) = 0 when 


w= 1/we—22p? or w=0. 


If we — 2p? is positive, then there is a positive value of w, namely w = \/w% — 2p? where the 
amplitude attains a maximum value. Since we know that the amplitude is Fo/(mw@) or Fo/k 
when w = 0, the maximum will be larger than this. As described above, this value, Fo/k is 
the expected amplitude, that is, the amplitude you would get with no oscillation, so that 
if the amplitude is larger than this for some value of w, this means that the oscillation at 
frequency w is resonating with the system to create a larger oscillation. This is the idea 
of practical resonance. It is practical because there is damping, so the situation is more 
physically relevant (to contrast with pure resonance), and still results in larger amplitudes of 
oscillation. 

Our previous work indicates that a system will exhibit practical resonance for some values 
of w whenever wi — 2p? is positive, and the frequency where the amplitude hits the maximum 
value is at \/w@ — 2p. This follows by the first derivative test for example as then C’(w) > 0 
for small w in this case. If on the other hand w? — 2p? is not positive, then C(w) achieves its 
maximum at w = 0, and there is no practical resonance since we assume w > 0 in our system. 
In this case the amplitude gets larger as the forcing frequency gets smaller. 
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If practical resonance occurs, the peak frequency is smaller than wo. As the damping y 
(and hence p) becomes smaller, the peak practical resonance frequency goes to wo. So when 
damping is very small, wo is a good estimate of the peak practical resonance frequency. This 
behavior agrees with the observation that when y = 0, then w9 is the resonance frequency. 

Another interesting observation to make is that when w — oo, then C > 0. This means 
that if the forcing frequency gets too high it does not manage to get the mass moving in 
the mass-spring system. This is quite reasonable intuitively. If we wiggle back and forth 
really fast while sitting on a swing, we will not get it moving at all, no matter how forceful. 
Fast vibrations just cancel each other out before the mass has any chance of responding by 
moving one way or the other. 

The behavior is more complicated if the forcing function is not an exact cosine wave, but 
for example a square wave. A general periodic function will be the sum (superposition) of 
many cosine waves of different frequencies. The reader is encouraged to come back to this 
section once we have learned about the ideas of Fourier series. 


2.6.3. Exercises 


Exercise 2.6.1: Write cos(3x) — cos(2x) as a product of two sine functions. 
Exercise 2.6.2: Write cos(5x) — cos(3x) as a product of two sine functions. 
Exercise 2.6.3: Write cos(3x) — cos(7x) as a product of two sine functions. 


Exercise 2.6.4: Derive a formula for x,, if the equation is mz" + ya' + kx = Fosin(wt). 
Assume y > 0. 


Exercise 2.6.5: Derive a formula for x,, if the equation is mx" + ya' + kx = Fo cos(wt) + 
F, cos(3wt). Assume y > 0. 


Exercise 2.6.6:* Derive a formula for x, for mx" + ya' + kx = Fo cos(wt) + A, where A is 
some constant. Assume y > 0. 


Exercise 2.6.7: Take mx" +-ya' +kax = Focos(wt). Fixm > 0, k > 0, and Fo > 0. Consider 
the function C(w). For what values of y (solve in terms of m, k, and Fy) will there be no 
practical resonance (that is, for what values of y is there no maximum of C(w) for w > 0)? 


Exercise 2.6.8: Take mx" + a'+kx = Focos(wt). Fix y > 0, k > 0, and Fo > 0. Consider 
the function C(w). For what values of m (solve in terms of y, k, and Fo) will there be no 
practical resonance (that is, for what values of m is there no maximum of Cw) for w > 0)? 


Exercise 2.6.9:* A mass of 4kg on a spring with k = 4N/m and a damping constant 
c= 1Ns/m. Suppose that Fy = 2N. Using forcing function Fo cos(wt), find the w that causes 
the maximum amount of practical resonance and find the amplitude. 


Exercise 2.6.10: An infant is bouncing in a spring chair. The infant has a mass of 8 kg, 
and the chair functions as a spring with spring constant 72N/m. The bouncing of the infant 
applies a force of the form 3cos(wt) for some frequency w. Assume that the infant starts at 
rest at the equilibrium position of the chair. 
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a) If there is no dampening coefficient, what frequency would the infant need to force at 
in order to generate pure resonance? 


b) Assume that the chair is built with a dampener with coefficient 5N%/m. Set up an initial 
value problem for this situation if the child behaves in the same way. 


c) Solve this initial value problem. 


d) There are several options for chairs you can buy. There is the one with dampening 
coefficient 5Ns/m, one with 1Ns/m, and one with 20Ns/m. Which of these would be most 
‘fun’ for the infant? How do you know? 


Exercise 2.6.11: A water tower in an earthquake acts as a mass-spring system. Assume 
that the container on top is full and the water does not move around. The container then acts 
as the mass and the support acts as the spring, where the induced vibrations are horizontal. 
The container with water has a mass of m = 10,000kg. It takes a force of 1000 newtons 
to displace the container 1 meter. For simplicity assume no friction. When the earthquake 
hits the water tower is at rest (it is not moving). The earthquake induces an external force 
F(t) = mAw? cos(wt). 


a) What is the natural frequency of the water tower? 


b) If w is not the natural frequency, find a formula for the maximal amplitude of the 
resulting oscillations of the water container (the maximal deviation from the rest 
position). The motion will be a high frequency wave modulated by a low frequency 
wave, so simply find the constant in front of the sines. 


c) Suppose A = 1 and an earthquake with frequency 0.5 cycles per second comes. What 
is the amplitude of the oscillations? Suppose that if the water tower moves more than 
1.5 meter from the rest position, the tower collapses. Will the tower collapse? 


Exercise 2.6.12:* Suppose there is no damping in a mass and spring system with m = 5, 
k = 20, and fp = 5. Suppose w is chosen to be precisely the resonance frequency. 


a) Find w. 


b) Find the amplitude of the oscillations at time t = 100, given the system is at rest at 
=20, 


Exercise 2.6.13: Assume that a 2 kg mass is attached to a spring that is acted on by a 
forcing function F(t) = 5cos(2t). Assume that there is no dampening on the spring. 


a) What should the spring constant k be in order for this system to exhibit pure resonance? 


b) If we wanted the system to exhibit practical resonance instead, what do or can we 
change about it to get this? 


c) Assume that we set k to be the value determined in (a), and that the rest of the problem 
is situated so that the system exhibits practical resonance. What would we expect to 
see for the amplitude of the solution? This should be a generic comment, not a specific 
value. 
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Exercise 2.6.14: Assume that we have a mass-on-a-spring system defined by the equation 
3y” + 2y' + 18y = 4cos(5t). 
a) Identify the mass, dampening coefficient, and spring constant for the system. 


b) Use the entire equation to find the natural frequency, forcing frequency, and quasi- 
frequency of this oscillation. 


c) Two of these frequencies will show up in the general solution to this problem. Which 
are they, and in which part (transient, steady-periodic) do they appear? 


d) Find the general solution of this problem. 


Exercise 2.6.15: A circuit is built with an L Henry inductor, and R Ohm resistor, and a 
C Farad capacitor. All of the units are correct, but you do not know any of their values. To 
study this circuit, you apply an external voltage source of F(t) = 4cos (4t), and the circuit 
starts with no initial charge or current. 


a) Write an initial value problem to model this situation. 


b) Your friend (who knows more about this circuit than you do) takes a reading from this 
circuit after it is running and says “The amplitude of the charge oscillation is greater 
than 100 coulombs, which means this circuit is exhibiting practical resonance.” There 
are three facts that you can learn about this circuit from the statement here that will 
tell you about the values of L, R, and C. 


(i) This statement seems to imply that the expected amplitude of the oscillation is 
100 coulombs. What does this mean about the value of C'? 


(ii) Your friend says that this circuit is in practical resonance. What does this tell 


you about the value of R in this case? 


(iii) Finally, being in practical resonance says something about how the forcing fre- 
quency compares to the natural frequency of this system. What is that, and how 
does it relate to the value of L? 


c) What is the frequency of the steady-periodic oscillation that your friend mentioned 
above? 
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2.7 Higher order linear ODEs 


Attribution: [JL], §2.8. 


After this section, you will be able to: 


e Find the general solution to a linear, constant coefficient, homogeneous differential 
equation of higher order and 

e Solve non-homogeneous higher order equations using the method of undetermined 
coefficients. 


In this section, we will briefly study higher order equations. Equations appearing in 
applications tend to be second order. Higher order equations do appear from time to time, 
but generally the world around us is “second order.” 

The basic results about linear ODEs of higher order are essentially the same as for second 
order equations, with 2 replaced by n. The important concept of linear independence is 
somewhat more complicated when more than two functions are involved. For higher order 
constant coefficient ODEs, the methods developed are also somewhat harder to apply, but 
we will not dwell on these complications. It is also possible to use the methods for systems of 
linear equations from chapter 4 to solve higher order constant coefficient equations. 

Let us start with a general homogeneous linear equation 


yg + Paa(a)y?? +--+ + pi(x)y + po(x)y = 0. (2.10) 
Theorem 2.7.1 (Superposition) 
Suppose yj, Y2,---, Yn are solutions of the homogeneous equation (2.10). Then 
y(@) = Ciys(@) + Coya(a) +--+ + Cnn) 


also solves (2.10) for arbitrary constants C1, C2,...,Cn. 


In other words, a linear combination of solutions to (2.10) is also a solution to (2.10). We 
also have the existence and uniqueness theorem for nonhomogeneous linear equations. 


Theorem 2.7.2 (Existence and uniqueness) 


Suppose po through p,_;, and f are continuous functions on some interval J, a is a 
number in J, and bo, bi,...,bn-1 are constants. The equation 


y™ + pr—r(z)y") +--+ + pi(x)y’ + po(x)y = f(x) 


has exactly one solution y(xz) defined on the same interval J satisfying the initial 
conditions 
y(a)=b, y(a)=b, ..., y (a) = dn-1. 
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2.7.1 Linear independence 


When we had two functions y; and yo we said they were linearly independent if one was not 
the multiple of the other. Same idea holds for n functions. In this case, it is easier to state 
as follows. The functions y1, yo, ..., Yn are linearly independent if the equation 


C1y1 + Coye +++ + CnYn = 0 


has only the trivial solution cy = co = --- = c, = 0, where the equation must hold for all x. 
If we can solve equation with some constants where for example c; 4 0, then we can solve for 
yi as a linear combination of the others. If the functions are not linearly independent, they 
are linearly dependent. 


Example 2.7.1: Show that e”, e?”, e®* are linearly independent. 


Solution: Let us give several ways to show this fact. Many textbooks (including [EP] and 
[F]) introduce Wronskians for higher order equations, but it is harder to analyze them without 
tools from linear algebra (see Chapter 3). Once there are more than two functions involved, 
there is not a nice, simple formula for the Wronskian (like yj y2 — ySy1 for two functions) and 
linear algebra is required to analyze what is happening here. Instead, we will take a slightly 
different and more improvized approach to see why these functions are linearly independent. 
Let us write down 
ce” + coe” + c3e** = 0. 


We use rules of exponentials and write z = e*. Hence z? = e?” and z? = e**. Then we have 
cz + te" + 32° = 0. 


The left-hand side is a third degree polynomial in z. It is either identically zero, or it has at 
most 3 zeros. Therefore, it is identically zero, cy = co = cz = 0, and the functions are linearly 
independent. 

Let us try another way. As before we write 


ae + oe" + ae = 0. 
This equation has to hold for all x. We divide through by e®” to get 
ce " +oc "+e =0. 


As the equation is true for all x, let x + oo. After taking the limit we see that cz; = 0. Hence 
our equation becomes 
cje" + coe” = 0. 


Rinse, repeat! 
How about yet another way. We again write 


ge oe + me" = 0. 


We can evaluate the equation and its derivatives at different values of x to obtain equations 
for C1, C2, and c3. Let us first divide by e* for simplicity. 


cy + coe” + c3e7” = 0. 
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We set x = 0 to get the equation c; + co + c3 = 0. Now differentiate both sides 
coe” + 2c3e7* = 0. 


We set x = 0 to get co + 2c3 = 0. We divide by e” again and differentiate to get 2c3e” = 0. 
It is clear that cz is zero. Then cy must be zero as Cp = —2c3, and c; must be zero because 
Cy + Co +63 = 0. 

There is no one best way to do it. All of these methods are perfectly valid. The important 
thing is to understand why the functions are linearly independent. | 


Exercise 2.7.1 (not necessary on first reading): Here is the linear algebra method for after 
reading through that chapter. Let y; = e”, y2 = e?* and y3 = e®”. Verify that 


yi (0) 1 y2(0) 1 y3(0) 1 
y(0)} = {1 y3(0)} = |2 y3(0)} = 3 
yi (0) 1 ys (0) 4 y3 (0) 9 


and use that to determine that these functions are linearly independent by showing that 
111 
dep} 1. <2)-3) =2 20 
14 9 
so that this matrix is invertible. 


Example 2.7.2: On the other hand, the functions e”, e~”, and cosh are linearly dependent. 
Simply apply definition of the hyperbolic cosine: 


2. —= 2. 
e +e = 
cosh = ——5— or 2coshaz —e* —e * =0. 


This second form here is a linear combination (coefficients 2, —1, and —1) of the three 
functions that adds to zero. 


2.7.2 Constant coefficient higher order ODEs 


When we have a higher order constant coefficient homogeneous linear equation, the song and 
dance is exactly the same as it was for second order. We just need to find more solutions. If 
the equation is n* order, we need to find n linearly independent solutions. It is best seen by 
example. 


Example 2.7.3: Find the general solution to 


mW 


y — 3y" —y' + 3y =0. (2.11) 
Solution: Try: y =e”. We plug in and get 


oot 2 72 rx rx 
ree —3r°e" —re+3.e" =0. 
wd 


yl! y” y! y 
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We divide through by e™. Then 
r? — 3r?-r+3=0. 


The trick now is to find the roots. There is a formula for the roots of degree 3 and 4 
polynomials but it is very complicated. There is no formula for higher degree polynomials. 
That does not mean that the roots do not exist. There are always n roots for an n“ degree 
polynomial. They may be repeated and they may be complex. Computers are pretty good at 
finding roots approximately for reasonable size polynomials. 

A good place to start is to plot the polynomial and check where it is zero. We can also 
simply try plugging in. We just start plugging in numbers r = —2,—1,0,1,2,... and see if 
we get a hit (we can also try complex numbers). Even if we do not get a hit, we may get an 
indication of where the root is. For example, we plug r = —2 into our polynomial and get 
—15; we plug in r = 0 and get 3. That means there is a root between r = —2 and r = 0, 
because the sign changed. If we find one root, say r1, then we know (r —1;) is a factor of our 
polynomial. Polynomial long division can then be used. 

Another technique for guessing roots of polynomials is the Rational Roots Theorem, which 
says that any rational root of the polynomial must be of the form p/q where p divides the 
constant term of the polynomial and q divides the leading term, provided neither of them are 
zero. For more information on this see § B.1. In this case, we would know that p must divide 
3, and g must divide 1. Therefore, the only possible options here are +1 and +3. These 
would be good places to start to look for rational roots. 

A good strategy is to begin with r = 0, 1, or —1. These are easy to compute. Our 
polynomial has two such roots, r; = —1 and rg = 1. There should be 3 roots and the last 
root is reasonably easy to find. The constant term in a monic* polynomial such as this is the 
multiple of the negations of all the roots because r? — 3r? —r +3 = (r—r1)(r —1re)(r — 73). 


So 
3 = (—11)(—Tr2)(—r3) = (1)(-1)(-73) = 73. 


You should check that r3 = 3 really is a root. Hence e~”, e” and e** are solutions to (2.11). 


They are linearly independent as can easily be checked, and there are 3 of them, which 
happens to be exactly the number we need. So the general solution is 


Y= Cie” + Cre” + Cre”. 


Another possible way to work out this general solution is by factoring the original 
polynomial. Since we want to solve 


r>? — 3r?7-r+3=0, 
we can rewrite the polynomial as 
r(r—3) —1(r—3) =0 


which factors as 
(r? —1)(r — 3) =0. 


d 


*The word monic means that the coefficient of the top degree r@, in our case r°, is 1. 
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Finally, using difference of two squares on the first factor gives 
(r—1)(r+1)(r-—3) =0. 


This gives roots of 1, —1, and 3, and so the same general solution as above. 
Suppose we were given some initial conditions y(0) = 1, y’(0) = 2, and y’"(0) = 3. Then 
1 = y(0) =C,+C,+ Cs, 
2= y'(0) = —C; + Cy + 3C3, 
3= y' (0) = Ci + C5 + 9C3. 
It is possible to find the solution by high school algebra, but it would be a pain. The sensible 
way to solve a system of equations such as this is to use matrix algebra, see § 4.2 or Chapter 3. 


For now we note that the solution is Cy = —1/4, Cp = 1, and C3 = 1/4. The specific solution 
to the ODE is 


ee eee ae 
=> € é = Gr’ 3 
ae 4 


| 


Next, suppose that we have real roots, but they are repeated. Let us say we have a root r 
repeated k times. In the spirit of the second order solution, and for the same reasons, we 


have the solutions 


k-1 
a ge, ge, uy, Oe 


We take a linear combination of these solutions to find the general solution. 


Example 2.7.4: Solve 
y) _ 3y"" a 3y" 2 y! =: 


Solution: We note that the characteristic equation is 
fp =3P 23 =r =. 


By inspection we note that r+ — 3r° 4+ 3r? —r = r(r— i)”, Hence the roots given with 
multiplicity are r = 0,1,1,1. Thus the general solution is 


y=(Ci+Con+Cz3r7)e"+ Cy. 
—-_-eswr ee; eo” ~— 
terms coming from r=1 from r=0 
Example 2.7.5: Find the general solution of 
y" + 2Qy" _ by _ 6y —_ 0 
Solution: The characteristic equation for this example is 
r? + 2r? — 5r —6=0. 


There is no convenient factoring by grouping or other quick formula to get to the roots here. 
The best hope we have is to try to guess the roots and see if we come up with anything. Once 
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we get one root, we'll be able to factor a term out and get down to a quadratic equation, 
where the quadratic formula will give us the other two roots. 

The properties of polynomials tell us that all rational roots of this polynomial must be 
factors of =$ or —6. Thus, the options are +1, +2, and +3. At this point, the best bet 
is to start guessing and see if we can find one. Let’s start with 1. Plugging this into the 
polynomial gives 

1° + 2(1)? — 5(1) -6 = -8 £0. 
Trying —1 next, we get 
(—1)? + 2(—1)? — 5(-1) -6 = -14+.24+5-6=0. 
Therefore, —1 is as root, and so (r + 1) is a factor of this polynomial. 
We can then use synthetic (or long) division to see that 


re +r? —5r—6=(r4+1)\(r? +r —6). 


For the quadratic, we can either use the quadratic formula, or just recognize that this 
factors as (r — 2)(r + 3) to get that the characteric equation factors as 


(r+1)(r —2)(r +3) =0. 
Therefore, the roots are —1, 2 and —3, so that the general solution to the differential equation 
is 
y(x) = Cye™* + Cye** + Cze~*". 7 
For more information on synthetic division and finding roots of polynomials, see Ap- 
pendix B.1. 
The case of complex roots is similar to second order equations. Complex roots always 


come in pairs r = a +718. Suppose we have two such complex roots, each repeated k times. 
The corresponding solution is 


(Cot Cie +---+C,_12*"1) e® cos(Bx) + (Do + Dit + --- + Dy_az*") e® sin(Bz). 
where Co, ..., Cx_1, Do, ..., Dz_1 are arbitrary constants. 
Example 2.7.6: Solve 
yO — Ay” + 8y” — 8y' + 4y = 0. 
Solution: The characteristic equation is 
r* — 4r? + 8r? — 8r +4=0, 
(r? —2r + >. =), 
((r —1)? +1) =0. 
Hence the roots are 1 +7, both with multiplicity 2. Hence the general solution to the ODE is 
y = (Ci + Cox) e* cosx + (C3 + Cyr) e* sin z. 


The way we solved the characteristic equation above is really by guessing or by inspection. It 
is not so easy in general. We could also have asked a computer or an advanced calculator for 
the roots. | 
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2.7.3. Non-Homogeneous Equations 


Just like for second order equation, we can solve higher order non-homogeneous equations. 
The theory is the same; if we can find any single solution to the non-homogeneous problem, 
then the general solution of the non-homogeneous problem is this single solution plus the 
general solution to the corresponding homogeneous problem. The trick comes down to finding 
this single solution, and undetermined coefficients is the main method here. 

In using undetermined coefficients, the guesses we want to make are the same as for second 
order equations. The only way it really gets more complicated is that now it is possible for 
any exponential or trigonometric function to be a solution to the homogeneous problem, and 
so more things will need to be multiplied by x in order to get the appropriate guess for the 
non-homogeneous solution. 


Example 2.7.7: Find the general solution to 
yn + 2y" _ By’ an 6y = 362% + e*. 


Solution: We found the general solution of the homogeneous problem in Example 2.7.5, 
which is 
y(x) = Cye™™ + Coe** + Cze7*". 
Now, to solve the non-homogeneous problem, we use the method of undetermined coeffi- 
cients. Since the non-homogeneous part of the equation has terms of the form e?* and e*, 


we would want to guess 
Yp(x) = Ae** + Be*. 


However, e?” solves the homogeneous problem, so we need to multiply it by 2, making our 
actual guess become 
Yp(x) = Are + Be™. 


In order to plug this in, we need to take three derivatives of this guess, which are 


Yp(xz) = Axe” + Be 

y,(x) = Ae” + 2Axe* + 4Be™ 

yp (a) = 4Ae** + 4Are*” + 16Be™ - 
yp (v) = 12Ae** + 8Are** + 64Be** 


By putting this into the non-homogeneous equation we want to solve, we get 


(12Ae?* + 8Are** + 64Be**) + 2(4Ac?* + 4Axe** + 16Be**) 
— 5(Ae™ + 2Are* + 4Be™) —6(Are” + Be™) = 3e" +e” 


Simplifying the left hand side of this expression gives 
15Ae* + 70Be™ = Se" +e”. 


To satisfy this equation, we want to set A = : and B = a: Therefore, the general solution 
to the non-homogeneous problem is 


1 1 
y(z) = Cie * + Coe” + Cae + sre" -- a 
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il 


Example 2.7.8: Determine the form of the guess using undetermined coefficients for finding 
a particular solution of the non-homogeneous problem 


yO 4 y® — ty — ay + y + y =e? + 3 + sin(x) + 2z. 


Solution: To determine the guess, we need to first find the solution to the homogeneous 
equations. The characteristic equation of the homogeneous equation is 


Gog? 07? Op pe =), 


We could use the root guessing method for this example, and all rational roots must be +1. 
However, that method is not great for polynomials that are of degree higher than around 3 
or 4. So, we’ll want to use some other technique to find all of the root. 

If we start by grouping pairs of terms, we can rewrite this polynomial as 


r(r+1)—2r4(r +1) +1(r+1) =0 
so that it can be rewritten as 
(r + 1)(r® — 2r* +1) =0. 
The second factor looks a lot like 
(s—1)?=5?-2s41 
if we take s = r+. Since 
($= 1) =(P 4G? - N= +H +DEO- 
using difference of squares twice. Thus, the entire characteristic equation can be written as 
(P+ 164-1)? = (+ YUP + OHO YP = (+ Yr — 20? +P. 


Therefore, we have a triple root at —1, a double root at 1, and two copies of (r? + 1), which 
has a root of i, corresponding to solutions sin(z) and cos(x). Putting all of this together, the 
general solution to the homogeneous equation is 


yo(x) = (Cy + Cox + Cax7\e™* + (C4 + Cyx)e” + (Cg + Cra) sin(x) + (Cg + Cox) cos(z). 


This has 9 unknown constants in it, which is expected from the ninth order equation. 

Now, we need to figure out the appropriate guess for the non-homogeneous solution. Since 
the non-homogeneous part of the equation is e” + 3e~* + sinx + 2x, the base guess would be 
of the form 

Ae* + Be“ +C sina + Deosx+ Ex+F 


because we always need to include both sin(x) and cos(x) whenever either of them appear. 
However, we need to factor in what terms show up in the homogeneous solution. For instance, 
the e* term has a term with 1 and x in the homogeneous solution, we need to include the 
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next one up in our guess for the solution to the non-homogeneous problem. Taking this into 
account for all terms gives the desired guess as 


Yp(x) = Axe” + Bu?e* + Cx? sin(x) + Dz? cos(x) + Ext F. 
: Z 


There is also an extension of variation of parameters to higher order equations. However, 
the fact that there are more terms in the solution means that the form of the expression is 
much more complicated that for second order, and is not worth looking into or trying to 
remember. The easier way to handle these situations using variation of parameters is by 
converting the higher order equation into a first order system and applying the methods 
there, which will be covered in § 4.1 and § 4.8 respectively. 


2.7.4 Exercises 


Exercise 2.7.2: Find the general solution for y/" — y" + y' —y = 0. 
Exercise 2.7.3:* Find the general solution of y® — y = 0. 
Exercise 2.7.4: Find the general solution for y“ — 5y'" + 6y" = 0. 
Exercise 2.7.5: Find the general solution for y'" + 2y” + 2y' = 0. 
Exercise 2.7.6: Suppose the characteristic equation for an ODE is (r — 1)*(r — 2)” = 0. 
a) Find such a differential equation. 
b) Find its general solution. 
Exercise 2.7.7: Suppose that a fourth order equation has a solution y = 2e**x cos x. 
a) Find such an equation. 
b) Find the initial conditions that the given solution satisfies. 


Exercise 2.7.8:* Suppose that the characteristic equation of a third order differential 
equation has roots +27 and 3. 


a) What is the characteristic equation? 

b) Find the corresponding differential equation. 

c) Find the general solution. 
Exercise 2.7.9: Find the general solution for the equation of Exercise 2.7.7. 
Exercise 2.7.10:* Find the general solution of 


yO — y” — By" — 23y' — 20y = 0. 
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Exercise 2.7.11: Find the general solution of 
y” — 6y" + 13y' — 10y = 4e” + 5e** — 20. 
Exercise 2.7.12: Find the general solution of 
yl” — 3y' + 2y = Qe” — e**. 

Exercise 2.7.13: Find the general solution of 

yf” + 2y" + y' + 2y = 3cos(z) + 2. 
Exercise 2.7.14: Find the general solution of 

y) + 2y" + y = 4a cos(x) — e®* +1 


Hint: Remember, the guess needs to make sure that no terms in it solve the homogeneous 
equation. 


Exercise 2.7.15: Show that y = cos(2t) is a solution to y“ + 2y'” + 9y” + 8y' + 20y = 0. 
This tells us something about the factorization of the characteristic polynomial of this DE. 
Factor the characteristic polynomial completely, and solve the DE. 


Exercise 2.7.16: Consider 
y —y" — 8y' + 12y = 0. (2.12) 


a) Show that y = e” is a solution of (2.12). 
b) Find the general solution to (2.12). 
c) Solve y" — y” — 8y' + 12y = e*. 


Exercise 2.7.17: Let f(x) = e* —cosz, g(x) = e*+cosz, and h(x) = cosa. Are f(x), g(x), 
and h(x) linearly independent? If so, show it, if not, find a linear combination that works. 


Exercise 2.7.18: Let f(x) = 0, g(x) = cosa, and h(x) = sing. Are f(x), g(x), and h(z) 
linearly independent? If so, show it, if not, find a linear combination that works. 


Exercise 2.7.19:* Are e”, e"*', e?”, sin(x) linearly independent? If so, show it, if not find 
a linear combination that works. 


Exercise 2.7.20: Are x, x”, and x* linearly independent? If so, show it, if not, find a linear 
combination that works. 


Exercise 2.7.21: Are e”, xe, and x’e* linearly independent? If so, show it, if not, find a 
linear combination that works. 


Exercise 2.7.22:* Are sin(x), x, xsin(x) linearly independent? If so, show it, if not find a 
linear combination that works. 


Exercise 2.7.23: Show that {e',te',e', te} is a linearly independent set. 
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Exercise 2.7.24:* Solve 1001y!" + 3.2y" + ry! — V4y = 0, y(0) = 0, y/(0) = 0, y’(0) = 0. 


Exercise 2.7.25: Could y = t? cost be a solution of a homogeneous DE with constant real 
coefficients? If so, give the minimum possible order of such a DE, and state which functions 
must also be solutions. If not, explain why this is impossible. 


Exercise 2.7.26: Find a linear DE with constant coefficients whose general solution is 
y = cye™ + cge* cos(4t) + cge~* sin(2t), 

or explain why there is no such thing. 

Exercise 2.7.27: Find an equation such that y = xe~*" sin(3z) is a solution. 


Exercise 2.7.28:* Find an equation of minimal order such that y = cos(x), y = sin(2), 
y = e* are solutions. 


Exercise 2.7.29: Find an equation of minimal order such that y = cos(x), y = sin(2z), 
y = e®” are solutions. 


Exercise 2.7.30: Find a homogeneous DE with general solution 


y= cje’ + ce’ +c3cost + cysint + cste’ + cgte’ + crt cost + cgt sint. 
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Chapter 3 


Linear algebra 


3.1 Vectors, mappings, and matrices 


Attribution: [JL], §A.1. 


After this section, you will be able to: 


e Express n-tuples of numbers as vectors, 


e Perform operations on vectors, and 
e Understand how linear maps on vectors give rise to matrices. 


In real life, there is most often more than one variable. We wish to organize dealing 
with multiple variables in a consistent manner, and in particular organize dealing with 
linear equations and linear mappings, as those both rather useful and rather easy to handle. 
Mathematicians joke that “to an engineer every problem is linear, and everything is a matrix.” 
And well, they (the engineers) are not wrong. Quite often, solving an engineering problem is 
figuring out the right finite-dimensional linear problem to solve, which is then solved with 
some matrix manipulation. Most importantly, linear problems are the ones that we know how 
to solve, and we have many tools to solve them. For engineers, mathematicians, physicists, 
and anybody in a technical field it is absolutely vital to learn linear algebra. 

As motivation, suppose we wish to solve 


L—y = 2, 
22 +y=4, 


for x and y, that is, find numbers x and y such that the two equations are satisfied. Let us 
perhaps start by adding the equations together to find 


e+2x-—yty=24+4, or JG = 6 


In other words, « = 2. Once we have that, we plug in x = 2 into the first equation to find 
2—y=2,s0 y= 0. OK, that was easy. What is all this fuss about linear equations. Well, 
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try doing this if you have 5000 unknowns*. Also, we may have such equations not of just 
numbers, but of functions and derivatives of functions in differential equations. Clearly we 
need a more systematic way of doing things. A nice consequence of making things systematic 
and simpler to write down is that it becomes easier to have computers do the work for us. 
Computers are rather stupid, they do not think, but are very good at doing lots of repetitive 
tasks precisely, as long as we figure out a systematic way for them to perform the tasks. 


3.1.1 Vectors and operations on vectors 


Consider n real numbers as an n-tuple: 


(1,2, wat POND 


The set of such n-tuples is the so-called n-dimensional space, often denoted by R”. Sometimes 
we call this the n-dimensional euclidean space'. In two dimensions, R? is called the cartesian 
plane, and in three dimensions, it is the same “3-dimensional space” that is dealt with in 
multivariable calculus. Each such n-tuple represents a point in the n-dimensional space. For 
example, the point (1,2) in the plane R? is one unit to the right and two units up from the 
origin. 

When we do algebra with these n-tuples of numbers we call them vectors’. Mathematicians 
are keen on separating what is a vector and what is a point of the space or in the plane, and 
it turns out to be an important distinction, however, for the purposes of linear algebra we 
can think of everything being represented by a vector. A way to think of a vector, which is 
especially useful in calculus and differential equations, is an arrow. It is an object that has a 
direction and a magnitude. For example, the vector (1,2) is the arrow from the origin to the 
point (1,2) in the plane. The magnitude is the length of the arrow. See Figure 3.1 on the 
facing page. If we think of vectors as arrows, the arrow doesn’t always have to start at the 
origin. If we do move it around, however, it should always keep the same direction and the 
same magnitude. 

As vectors are arrows, when we want to give a name to a vector, we draw a little arrow 
above it: 


> 


x 


Another popular notation is x, although we will use the little arrows. It may be easy to write 
a bold letter in a book, but it is not so easy to write it by hand on paper or on the board. 
Mathematicians often don’t even write the arrows. A mathematician would write x and just 
remember that x is a vector and not a number. Just like you remember that Bob is your 
uncle, and you don’t have to keep repeating “Uncle Bob” and you can just say “Bob.” In 
this book, however, we will call Bob “Uncle Bob” and write vectors with the little arrows. 


*One of the downsides of making everything look like a linear problem is that the number of variables 
tends to become huge. 

‘Named after the ancient Greek mathematician Euclid of Alexandria (around 300 BC), possibly the most 
famous of mathematicians; even small towns often have Euclid Street or Euclid Avenue. 

*Named after the French mathematician René Descartes (1596-1650). It is “cartesian” as his name in 
Latin is Renatus Cartesius. 

8A common notation to distinguish vectors from points is to write (1,2) for the point and (1,2) for the 
vector. We write both as (1,2). 
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Figure 3.1: The vector (1,2) drawn as an arrow from the origin to the point (1,2). 


The magnitude can be computed using Pythagorean theorem. The vector (1,2) drawn 
in the figure has magnitude V1? + 22 = 5. The magnitude is denoted by ||#||, and, in any 
number of dimensions, it can be computed in the same way: 


2] = (er, 22)... 2tm)|] = fa Hak $22. 


For reasons that will become clear in the next section, we often write vectors as so-called 
column vectors: 
Ban 
v2 


In 


Don’t worry. It is just a different way of writing the same thing, and it will be useful later. 
For example, the vector (1,2) can be written as 


BF 


The fact that we write arrows above vec- 
tors allows us to write several vectors 1, 
Xo, etc., without confusing these with the + 
components of some other vector 2. 


So where is the algebra from linear alge- era ‘. 
bra? Well, arrows can be added, subtracted, ye Lm 
and multiplied by numbers. First we consider 0 0 ra CS ae 
addition. If we have two arrows, we simply RY ; 


move along one, and then along the other. 11 
See Figure 3.2. 


It is rather easy to see what it does to the 
numbers that represent the vectors. Suppose 
we want to add (1,2) to (2,—3) as in the 
figure. So we travel along (1,2) and then we 
travel along (2,—3) in the sense of “tip-to- 


Figure 3.2: Adding the vectors (1,2), drawn 
dotted, and (2,—3), drawn dashed. The result, 
(3,1), ts drawn as a solid arrow. 


tail” addition that you may have seen in previous classes. What we did was travel one unit 
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right, two units up, and then we travelled two units right, and three units down (the negative 
three). That means that we ended up at (1+2,2+(-—3)) = (3,—1). And that’s how addition 


always works: 


Ly Y1 
nop) Y2 

7 : 
Ln, Un 


Subtracting is similar. What x — y means 
visually is that we first travel along 7, and 
then we travel backwards along 7. See Fig- 
ure 3.3. It is like adding z+ (—y’) where —7 
is the arrow we obtain by erasing the arrow 
head from one side and drawing it on the 
other side, that is, we reverse the direction. 
In terms of the numbers, we simply go back- 
wards in both directions, so we negate both 
numbers. For example, if 7 is (—2,1), then 
~¥ is (2, —1). 


TT Yi 

t+ Y2 

Ln + Yn 
rok 


Figure 3.3: Subtraction, the vector (1,2), 
drawn dotted, minus (—2,1), drawn dashed. 
The result, (3,1), is drawn as a solid arrow. 


Another intuitive thing to do to a vector 
is to scale it. We represent this by multipli- 
cation of a number with a vector. Because of this, when we wish to distinguish between 
vectors and numbers, we call the numbers scalars. For example, suppose we want to travel 
three times further. If the vector is (1,2), travelling 3 times further means going 3 units to 
the right and 6 units up, so we get the vector (3,6). We just multiply each number in the 
vector by 3. If a is a number, then 


Uy AL, 

v2 AX 
a = A 

Xn AL 


Scaling (by a positive number) multiplies the magnitude and leaves direction untouched. The 
magnitude of (1,2) is /5. The magnitude of 3 times (1,2), that is, (3,6), is 35. 

When the scalar is negative, then when we multiply a vector by it, the vector is not only 
scaled, but it also switches direction. So multiplying (1,2) by —3 means we should go 3 times 
further but in the opposite direction, so 3 units to the left and 6 units down, or in other 
words, (—3, —6). As we mentioned above, —y is a reverse of ¥, and this is the same as (—1)y. 

In Figure 3.4 on the next page, you can see a couple of examples of what scaling a vector 
means visually. 

We put all of these operations together to work out more complicated expressions. Let us 
compute a small example: 


ef) - penta - 
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—1.52% 
Qt 
ff 


Figure 8.4: A vector %, the vector 2% (same direction, double the magnitude), and the vector 
—1.5% (opposite direction, 1.5 times the magnitude). 


As we said a vector is a direction and a magnitude. Magnitude is easy to represent, it is 
just a number. The direction is usually given by a vector with magnitude one. We call such 
a vector a unit vector. That is, @ is a unit vector when ||u|| = 1. For example, the vectors 
(1,0), @/v2, /v2), and (0, —1) are all unit vectors. 

To represent the direction of a vector 7, we need to find the unit vector in the same 
direction. To do so, we simply rescale x by the reciprocal of the magnitude, that is Take or 


> 


more concisely la 


For example, the unit vector in the direction of (1,2) is the vector 
L439) ( 1 2 ) 


3.1.2 Matrices 


The next object we need to define here is a matriz. 


Definition 3.1.1 


In general, an m x n matrix A is a rectangular array of mn numbers, 


Qm1 Gm2 ‘°° Amn 


An m X n matrix indicates that it will have m rows and n columns. 


Matrices, just like vectors, are generally written with square brackets on the outside, 
although some books will use parentheses for this. The convention for notation is that 
matrices will be denoted by capital letters (A) and the individual entries of the matrix, the 
numbers that make it up, will be denoted using lowercase letters (a;;) where the first number 
7 indicates which row of the matrix we are talking about, and the second number 7 indicates 
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which column. For example, in the matrix 


1 ao 
A= |—9 3 1! , 
2 
we could talk about the entire matrix usint A, but would also have that a2; = —2 and a33 = 5. 
Note that an m x 1 matrix is just a column vector, so in terms of the basic structure, 
matrices are an extension of vectors. However, they can be used for so much more, as we will 
see in future sections. 
Another way to view matrices is as a set of column vectors all laid out side-by-side. If we 
have U1, V2 and v3, three different four component vectors, we can form a 4 x 3 matrix B as 


B= |v, | v2 | 3] 


that uses each of the given vectors as a column of the matrix. In this case, the vertical lines 
are used to indicate that this is actually a matrix, because each of the entries given there are 
vectors, not just individual numbers. If we wanted to write a 1 x 3 matrix this way, these 
vertical lines will not be included. 

We will go into more properties of matrices and the operations we can perform on them 
in § 3.2. To conclude this section though, we will look at one other way that matrices come 
about, and that is as the representation of a linear map. 


3.1.3. Linear mappings and matrices 


A vector-valued function F is a rule that takes a vector # and returns another vector 7. For 
example, F’ could be a scaling that doubles the size of vectors: 


F(Z) = 27. 


For example, 


If F is a mapping that takes vectors in R? to R? (such as the above), we write 


F: R? > R’. 


The words function and mapping are used rather interchangeably, although more often than 
not, mapping is used when talking about a vector-valued function, and the word function is 
often used when the function is scalar-valued. 
A beginning student of mathematics (and many a seasoned mathematician), that sees an 
expression such as 
f (3x + 8y) 


yearns to write 


3f(x) + 8f(y). 
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After all, who hasn’t wanted to write /z + y = x + ,/y or something like that at some 
point in their mathematical lives. Wouldn’t life be simple if we could do that? Of course we 
can’t always do that (for example, not with the square roots!) It turns out there are many 
functions where we can do exactly the above. Such functions are called linear. 

A mapping F’: R” > R” is called linear if 


P+ y) = F(@) + FY), 


for any vectors Z and y, and also 
F(az) = aF(2), 
for any scalar a. The F' we defined above that doubles the size of all vectors is linear. Let us 


check: 
F(@+y) =2(¢4+ ¥) = 224+ 27 = F(£) + Fly), 


and also 


Mog) = 2er = 020 =a (Ff 
We also call a linear function a linear transformation. If you want to be really fancy and 


impress your friends, you can call it a linear operator. 
When a mapping is linear we often do not write the parentheses. We write simply 


Px 


instead of F(Z). We do this because linearity means that the mapping F’ behaves like 
multiplying < by “something.” That something is a matrix. 

Now how does a matrix A relate to a linear mapping? Well a matrix tells you where 
certain special vectors go. Let’s give a name to those certain vectors. The standard basis 
vectors of IR” are 


1 0 0 0 
0 1 0 0 
é = |], é& = |], é; = | 1], eee 
0 0 0 1 


For example, in R? these vectors are 
1 0 0 
Ei = 0 5 Eo — 1 5 €3 = 0 
0 0 1 


> 


You may recall from calculus of several variables that these are sometimes called 7, 7, k. 
The reason these are called a basis is that every other vector can be written as a linear 
combination of them. For example, in R® the vector (4,5,6) can be written as 


1 0 0 4 
4e) + 5€) + 6€3 = 4 0] +5 ]1]) +640] = [5 
0 0 1 6 
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Keep this idea of linear combinations of vectors in mind; we'll see a lot more of it later. 
So how does a matrix represent a linear mapping? Well, the columns of the matrix are 
the vectors where A as a linear mapping takes €), €2, etc. For example, consider 


i ae 
M= . ale 
As a linear mapping /: R* > R? takes €; = [4] to [4] and &) = [9] to [2]. In other words, 


mente Allo bl) mt eof dhl 


More generally, if we have an n x m matrix A, that is we have n rows and m columns, 
then the mapping A: R” — R” takes €; to the j column of A. For example, 


Q11 412 413 Gj4 15 
A= |aa1 a22 a23 A24 425 
Q31 432 433 434 435 


represents a mapping from R° to R® that does 


ay 12 Q13 ay4 Q15 
Aé; = | G21] , Aéz = | G22} , Aé3 = | @23 ) Aé4 = | Go4 ; Aés, = | Gas 
31 32 33 34 35 


But what if I have another vector 7? Where does it go? Well we use linearity. First write 
the vector as a linear combination of the standard basis vectors: 


X41 1 0 0 0 0 
X2 0 1 0 0 0 
— @3| = 0 + 2X2 0 +23 1 + 2X4 0 + £5 OO; = rE) L€o 1363 + L4€4 4 Unes. 
L4 0 0 0 1 0 
is 0 0 0 0 1 
Then 


At = A(21€) + L2€g + L3€3 44 U4e4 Ls) = v1 Ae; + Ty A€> + 13 Ae€3 + 4 Ae, + r5Aés. 


If we know where A takes all the basis vectors, we know where it takes all vectors. 
As an example, suppose M is the 2 x 2 matrix from above, and suppose we wish to find 


—2 1 2] |-2 1 2 —1.8 
ue | re P A | 72 H are A 7 ie 
Every linear mapping from R™ to R” can be represented by an n x m matrix. You 
just figure out where it takes the standard basis vectors. Conversely, every n x m matrix 


represents a linear mapping. Hence, we may think of matrices being linear mappings, and 
linear mappings being matrices. 
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Or can we? In this book we study mostly linear differential operators, and linear 
differential operators are linear mappings, although they are not acting on R”, but on an 
infinite-dimensional space of functions: 


Lf=g 
for a function f we get a function g, and L is linear in the sense that 
L(f +h) =Lf +h, and Llaf)=alf. 


for any number (scalars) a and all functions f and h. 

So the answer is not really. But if we consider vectors in finite-dimensional spaces R” 
then yes, every linear mapping is a matrix. We have mentioned at the beginning of this 
section, that we can “make everything a vector.” That’s not strictly true, but it is true 
approximately. Those “infinite-dimensional” spaces of functions can be approximated by a 
finite-dimensional space, and then linear operators are just matrices. So approximately, this 
is true. And as far as actual computations that we can do on a computer, we can work only 
with finitely many dimensions anyway. If you ask a computer or your calculator to plot a 
function, it samples the function at finitely many points and then connects the dots*. It 
does not actually give you infinitely many values. So the way that you have been using the 
computer or your calculator so far has already been a certain approximation of the space of 
functions by a finite-dimensional space. 


3.1.4. Exercises 


Exercise 3.1.1: On a piece of graph paper draw the vectors: 
2 —2 
one ») 73] ©) (3,-4) 
Exercise 3.1.2: On a piece of graph paper draw the vector (1,2) starting at (based at) the 
given point: 
a) based at (0,0) b) based at (1, 2) c) based at (0,—1) 


Exercise 3.1.3: On a piece of graph paper draw the following operations. Draw and label 
the vectors involved in the operations as well as the result: 


za Pela » Ta] -B ost 


Exercise 3.1.4: Compute the magnitude of 


a) Hl b) 3 c) (1,3, -4) 


*In Matlab, you may have noticed that to plot a function, we take a vector of inputs, ask Matlab to 
compute the corresponding vector of values of the function, and then we ask it to plot the result. 
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Exercise 3.1.5:* Compute the magnitude of 


"fs 


Exercise 3.1.6: Compute 


” b+ LI 
vale 


Exercise 3.1.7:* Compute 


ofl 


oot a 


2 
b) | 3 


»G-E 
oof 
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c) (—2,1, —2) 


a-[3) 
9 2|2)-9[2, 


Exercise 3.1.8: Find the unit vector in the direction of the given vector 


0 |) 


2 
b) | 1 
—] 


c) (3,1, —2) 


Exercise 3.1.9:* Find the unit vector in the direction of the given vector 


» [a 


1 
eel 
2 


c) (2, —5, 2) 


Exercise 3.1.10: If # = (1,2) and ¥ are added together, we find ¢ + 7 = (0,2). What is y? 


Exercise 3.1.11: If ¢ = (1, —4,3) and w = (—2,3, —-1), compute 30 — 2W and 40 + Jv. 


Exercise 3.1.12: Write (1,2,3) as a linear combination of the standard basis vectors €1, €2, 


and €3. 


Exercise 3.1.13: If the magnitude of x is 4, what is the magnitude of 


a) 0 b) 3 


c) -£ d) —4z 


ce) +2 f) @-z 


Exercise 3.1.14:* If the magnitude of # is 5, what is the magnitude of 


a) 4¢ 


b) -2z 


Exercise 3.1.15: Suppose a linear mapping F': R? > 
(0,1) to (3,3). Where does it take 


c) —4z¢ 


R? takes (1,0) to (2,—1) and it takes 
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a) (1,1) b) (2,0) c) (2,1) 


Exercise 3.1.16: Suppose a linear mapping F': R® > R? takes (1,0,0) to (2,1) and it 
takes (0,1,0) to (3,4) and it takes (0,0,1) to (5,6). Write down the matrix representing the 
mapping F’. 


Exercise 3.1.17: Suppose that a mapping F': R? — R? takes (1,0) to (1,2), (0,1) to (3,4), 
and it takes (1,1) to (0,—1). Explain why F is not linear. 


Exercise 3.1.18:* Suppose a linear mapping F': R? — R? takes (1,0) to (1,—1) and it 
takes (0,1) to (2,0). Where does it take 


a) (1,1) b) (0, 2) ce) (1, -1) 


Exercise 3.1.19 (challenging): Let P represent the space of quadratic polynomials in t: a 
point (ag, 41,2) in P represents the polynomial ag + at + agt?. Consider the derivative a 


as a mapping of P to P, and note that a is linear. Write down £ as a3 x 3 matrix. 


dt 
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3.2. Matrix algebra 


Attribution: [JL], §A.2. 


After this section, you will be able to: 


e Perform addition and multiplication operations on matrices, 


e Compute inverses of 2 x 2 matrices, and 
e Identify triangular, diagonal, and symmetric matrices. 


3.2.1 One-by-one matrices 


Let us motivate what we want to achieve with matrices. What do real-valued linear mappings 
look like? A linear function of real numbers that you have seen in calculus is of the form 


f(z) =mz +b. 
However, the properties of linear mappings discussed in the previous section are that 


fle+y)=f@)+fy) flax) =af(z). 
Plugging in the definition from above gives that 


F(e+y) =m(e+y)+b=mz4+my+b 
flax) = m(axz) +b=a(ma) +b 


and neither of these match up appropriately, since 


f(z) + f(y) =me+b4+my+b=mz+my-+ 2b 
af (x) + a(ma + b) = a(max) + ab , 


In order for these to work, we need to have b = 0. Therefore, real-valued linear mappings of 
the real line, linear functions that eat numbers and spit out numbers, are just multiplications 
by a number. 

Consider a mapping defined by multiplying by a number. Let’s call this number a. The 
mapping then takes x to ax. What we can do is to add such mappings. If we have another 
mapping (@, then 

ax + Bx = (a+ B)z. 


We get a new mapping a+ @ that multiplies x by, well, a+. If D is a mapping that doubles 
things, Dx = 2x, and T is a mapping that triples, Ta = 32, then D+ T is a mapping that 
multiplies by 5, (D+ T)x = 5z. 

Similarly we can compose such mappings, that is, we could apply one and then the other. 
We take x, we run it through the first mapping a to get a times x, then we run ax through 
the second mapping (. In other words, 


Blax) = (Baja. 
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We just multiply those two numbers. Using our doubling and tripling mappings, if we double 
and then triple, that is T(Dx) then we obtain 3(2x) = 6x. The composition TD is the 
mapping that multiplies by 6. For larger matrices, composition also ends up being a kind of 
multiplication. 


3.2.2 Matrix addition and scalar multiplication 


The mappings that multiply numbers by numbers are just 1 x 1 matrices. The number a 
above could be written as a matrix [a]. So perhaps we would want to do the same things to 
all matrices that we did to those 1 x 1 matrices at the start of this section above. First, let 
us add matrices. If we have a matrix A and a matrix B that are of the same size, say m x n, 
then they are mappings from R” to R™. The mapping A+ B should also be a mapping from 
R” to R™, and it should do the following to vectors: 


(A+ B)# = At + Bz. 


It turns out you just add the matrices element-wise: If the ij‘ entry of A is aj;;, and the ij" 
entry of B is b;;, then the ij entry of A+ B is aj; + bij. If 


a= be a2 i aa i . bio sl 


Q21 422 43 bo: ba2 bag 


then 
Acie = & +b, di2+ die Q13 7 | 


dz1 +b a2 + b22 dag + box 


Let us illustrate on a more concrete example: 


ley 7 8 Ley 243 8 10 
3 444+ /9 10); = ]34+9 4410; = ]12 14 
5 6 Li Si 5+11 6-1 16 5 


Let’s check that this does the right thing to a vector. Let’s use some of the vector algebra 
that we already know, and regroup things: 


1 2] 7, G2 8 hes 1 2 v4 8 
3 4 2+ 9 10 He DiS ee hal | oO Oly 2226 
5 6 15 4) 5 6 i = 
1 7 2 8 
=2/ |/3}+/9|]-{ [4] 4 | 10 
5 11 6 =I 
(eee, D8 8 10 
=2]3+4+9]— }4+4+10] =2]12] — |14 
5+11 6-1 16 5 
8 10] ;, 2(8) — 10 6 
= |12 14 eA = |2(12)-—14] = |10 
16 5 2(16) — 5 oT 
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If we replaced the numbers by letters that would constitute a proof! You'll notice that we 
didn’t really have to even compute what the result is to convince ourselves that the two 
expressions were equal. 

If the sizes of the matrices do not match, then addition is not defined. If A is 3 x 2 and B 
is 2 x 5, then we cannot add these matrices. We don’t know what that could possibly mean. 


It is also useful to have a matrix that when added to any other matrix does nothing. This 
is the zero matrix, the matrix of all zeros: 


12 FOO) 2 
3 4 O.Oy 3 > Aye 
We often denote the zero matrix by 0 without specifying size. We would then just write 


A+0, where we just assume that 0 is the zero matrix of the same size as A. 


There are really two things we can multiply matrices by. We can multiply matrices by 
scalars or we can multiply by other matrices. Let us first consider multiplication by scalars. 
For a matrix A and a scalar @ we want aA to be the matrix that accomplishes 


(aA) = a(AZ). 


That is just scaling the result by a. If you think about it, scaling every term in A by a 
accomplishes just that: If 


Qi, G12 ay Qa11; AA\2 AA;3 
A= mits then aA = : 
Q21 422 493 Qa21 AA22 AA23 


sit 2 3]_[2 4 6 
As he 50) DOE al) 
Let us list some properties of matrix addition and scalar multiplication. Denote by 0 the 
zero matrix, by a, 9 scalars, and by A, B, C matrices. Then: 


For example, 


ALO= A=O2A 
AL BEBLA 
(ALB eC SA (Bac), 
a(A+ B)=aA+aB, 
(0+ B)A=0A4+8A. 


These rules should look very familiar. 


3.2.3. Matrix multiplication 


As we mentioned above, composition of linear mappings is also a multiplication of matrices. 
Suppose A is an m X n matrix, that is, A takes R” to R™, and B is an n x p matrix, that is, 
B takes R? to R". The composition AB should work as follows 


AB# = A(B2). 
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First, a vector ¢ in R? gets taken to the vector Bx in R”. Then the mapping A takes it to 
the vector A(BZ) in R™. In other words, the composition AB should be an m x p matrix. 
In terms of sizes we should have 


4 ” 


[m x n] [n x p] = [m x p]. 
Notice how the middle size must match. 

OK, now we know what sizes of matrices we should be able to multiply, and what the 
product should be. Let us see how to actually compute matrix multiplication. We start 
with the so-called dot product (or inner product) of two vectors. Usually this is a row vector 
multiplied with a column vector of the same size. Dot product multiplies each pair of entries 
from the first and the second vector and sums these products. The result is a single number. 
For example, 


[ay ag as| ; by = a,b + Agb + a3b3. 


And similarly for larger (or smaller) vectors. A dot product is really a product of two matrices: 
al xn matrix and an n x 1 matrix resulting in a 1 x 1 matrix, that is, a number. 

Armed with the dot product we define the product of matrices. First let us denote by 
row;(A) the i** row of A and by column,(A) the j column of A. For an m x n matrix A 
and an n xX p matrix B we can compute the product AB. The matrix AB is an m x p matrix 
whose ij'* entry is the dot product 


row;(A) - column,(B). 
For example, given a 2 x 3 and a 3 Xx 2 matrix we should end up with a 2 x 2 matrix: 


by, b 

a1, G12 413 oF tae =f 11041 + Ay2b21 + 443031 11012 + Gy2b22 + €13b32 
b 1 boo = 5 (3.1) 
b 


a21 422 423 b 91611 + agabe1 + a93b31 ag1b12 + aabo2 + 9339 
32 


or with some numbers: 


oo 


sci) = [a8 2 |) 


A useful consequence of the definition is that the evaluation Av for a matrix A and a 
(column) vector # is also matrix multiplication. That is really why we think of vectors as 
column vectors, or n x 1 matrices. For example, 


te OP ON es eee Cat. Ne 
Za} =ql| |g deed s(eny| lal 
If you look at the last section, that is precisely the last example we gave. 


You should stare at the computation of multiplication of matrices AB and the previous 
definition of Ay as a mapping for a moment. What we are doing with matrix multiplication 


or by 
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is applying the mapping A to the columns of B. This is usually written as follows. Suppose 
we write the n x p matrix B = [by by --- bp], where by, bo,...,b, are the columns of B. Then 
for an m xX n matrix A, 


AB = Alb, by «++ b)| = [Ab, Abp «++» Ady]. 


The columns of the m x p matrix AB are the vectors Abn, Abn, ia Aby. For example, in 


(3.1), the columns of 


F] 


bit bye 
a1, G12 413 b b 
e s 4 21 922 
21 22 23 
b31 — b32 
are 
bi Dip 
a1, G12 413 a4, 412 443 
b 
21 and boa 
a21 422 423 b a21 G22 423 i 
31 


This is a very useful way to understand what matrix multiplication is. It should also make it 
easier to remember how to perform matrix multiplication. 


3.2.4 Some rules of matrix algebra 


For multiplication we want an analogue of a 1. That is, we desire a matrix that just leaves 
everything as it found it. This analogue is the so-called identity matriz. The identity matrix 
is a Square matrix with 1s on the main diagonal and zeros everywhere else. It is usually 
denoted by J. For each size we have a different identity matrix and so sometimes we may 
denote the size as a subscript. For example, the J; would be the 3 x 3 identity matrix 


il eet 
l=iz= |0' 1 6 
001 
Let us see how the matrix works on a smaller example, 


Qi1 2) }l Of fau-L+ay-0  ay-OFay-1) _ fan ary 
421 22} |O 1 Qqa1-1+a2-0 = agq1-0+ag2-1 a2, A229] 
Multiplication by the identity from the left looks similar, and also does not touch anything. 


We have the following rules for matrix multiplication. Suppose that A, B, C’ are matrices 
of the correct sizes so that the following make sense. Let a denote a scalar (number). Then 


A(BC) = (AB)C 
A(B+C)=AB+AC 
(B+C)A=BA+CA 

a(AB) = (aA)B = A(aB), 

IA=A=AlI 


(associative law), 
(distributive law), 


(distributive law), 


(identity). 
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Example 3.2.1: Let us demonstrate a couple of these rules. For example, the associative 


33-8.) 7 (a: ta) Si aN [8% 3) 6341). | 06 78 
2. =o 1 —3}|5 2)/ | 2 2] |-16 -2} | 64 52/7 


A(BC) 


—3 3 4 4 SLA pao a2) bad Ay. | 96- — 78 
2 —2| |1 -3 Me, he 1 PG 14 Be Blo 164 pe. |, 
Ca ee ee eee ee ee Ne ee 
A B S: AB ie (AB)C 


Or how about multiplication by scalars: 


—-3 3, |4 4 Oe 2 —90 —210 
(Te Alb S))-e[e a] = [oo ao | 
ee ee ee ee ae eee 


B AB 10(AB) 


wl 3]\ [4 4] [-30 30] [4 4] _ [-90 —210 
Pi 20) 4 lL 38) 10: S90) 1-8, || 60: 140 
eee eee eee NSS 


10A B (10A)B 


-3 3](,,[4 4]\_[-3 3] [40 40] _ [-90 —210 
Os 22 1 -3})~ | 2 -2]|10 -30}~ | 60 140 | ° 
| —-—S —S_S >” —— n——_-—>_“_’ 


10B A(10B) 


and 


A multiplication rule you have used since primary school on numbers is quite conspicuously 
missing for matrices. That is, matrix multiplication is not commutative. Firstly, just because 
AB makes sense, it may be that BA is not even defined. For example, if A is 2 x 3, and B is 
3 x 4, the we can multiply AB but not BA. 

Even if AB and BA are both defined, does not mean that they are equal. For example, 
take A =[}}] and B=[§9]: 


of IRIE] + Ga-baG d- 


3.2.5 Inverse 


A couple of other algebra rules you know for numbers do not quite work on matrices: 
(i) AB = AC does not necessarily imply B = C, even if A is not 0. 


(ii) AB =0 does not necessarily mean that A = 0 or B= 0. 


bb =f l= b olf a 


For example: 


198 CHAPTER 3. LINEAR ALGEBRA 


To make these rules hold, we do not just need one of the matrices to not be zero, we 
would need to “divide” by a matrix. This is where the matrix inverse comes in. 


Definition 3.2.1 


Suppose that A and B are n x n matrices such that 


Als — i Ae 


Then we call B the inverse of A and we denote B by A7!. 
If the inverse of A exists, then we say A is invertible. If A is not invertible, we say A is 
singular. 


Perhaps not surprisingly, Ay = A, since if the inverse of A is B, then the inverse of 
Bis A. 

If A = [a] is a 1 x 1 matrix, then A~! is a~' = +. That is where the notation comes from. 
The computation is not nearly as simple when A is larger. 

The proper formulation of the cancellation rule is: 


If A is invertible, then AB = AC implies B=C. 


The computation is what you would do in regular algebra with numbers, but you have to be 
careful never to commute matrices: 


AB = AC, 
A“ AB = A-C, 
IB =IC, 
B=C. 


And similarly for cancellation on the right: 
If A is invertible, then BA =CA implies B=C. 


The rule says, among other things, that the inverse of a matrix is unique if it exists: If 
AB =I = AC, then A is invertible and B = C. 

We will see later how to compute an inverse of a matrix in general. For now, let us note 
that there is a simple formula for the inverse of a 2 x 2 matrix 


ab) 1 fd = 
ec dl  ad—be|—c al’ 


For example: 


Let’s try it: 


bq v= a mt Ea elb =o a) 
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Just as we cannot divide by every number, not every matrix is invertible. In the case of 
matrices however we may have singular matrices that are not zero. For example, 


4 


is a singular matrix. But didn’t we just give a formula for an inverse? Let us try it: 


ee | a Oe: ai 0) 
2 2)  1-2-1-2[-2 1] ° 
We get into a bit of trouble; we are trying to divide by zero. 
So a 2 x 2 matrix A is invertible whenever 


ad — bc #0 


and otherwise it is singular. The expression ad — bc is called the determinant and we will 
look at it more carefully in a later section. There is a similar expression for a square matrix 
of any size. 


3.2.6 Special types of matrices 


A simple (and surprisingly useful) type of a square matrix is a so-called diagonal matriz. It is 
a matrix whose entries are all zero except those on the main diagonal from top left to bottom 
right. For example a 4 x 4 diagonal matrix is of the form 


di 0 0 0 
0 d. 0 0 
0 0 dz 0 
0 0 0 d 


Such matrices have nice properties when we multiply by them. If we multiply them by a 
vector, they multiply the k‘" entry by d,. For example, 


1 0 OF {4 1-4 4 
O 2 O} 15) = ]2-5] = 110 
0 0 3] |6 3°6 18 


Similarly, when they multiply another matrix from the left, they multiply the k*® row by dg. 
For example, 

20 O bs ol« Zl 

0 3 O dt “dy. | i Se. Be  D 

0 0 -1] {1 1 1 —1 -1 -l 
On the other hand, multiplying on the right, they multiply the columns: 


ge Bae WW We a: | 
fe le POE Bs ON a ad 
Lt ods SA]. OF Os ak De sd 
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And it is really easy to multiply two diagonal matrices together: 


1 0 O] [2 0 O 1-2 O 0 2 0 O 
OF 20 Od. Os en), 0 2e3 0 = |0 6 O 
O08 call Os Oe ae 0 0 3-(-1) 0 0 —83 


For this last reason, they are easy to invert, you simply invert each diagonal element: 


de 00 Oe “fae oo 8 
Oradea, 0) asi ade 20 
0 0 ds QO. 20) tde* 
Let us check an example 
PAOOl. P20 00 + 0 0] [2 0 0 1 0 0 
0 3 0 O80 00-2 0); 10t 3 0) 1 0o- Pe 6 
0 0 4 0 0 4 00 4] [0 0 4 001 
—S SF —eSE OOS 
A-1 A A-1 A 


It is no wonder that the way we solve many problems in linear algebra (and in differential 
equations) is to try to reduce the problem to the case of diagonal matrices. 

Another type of matrix that has similarly nice properties are triangular matrices. A 
matrix is upper triangular if all of the entries below the diagonal are zero. For a 3 x 3 matrix, 
an upper triangular matrix looks like 


oo x 
Oo x * 
x * * 


where the * can be any number. Similarly, a lower triangular matrix is one where all of the 
entries above the diagonal are zero, or, for a 3 x 3 matrix, something that looks like 


0 0 
*« O 
x ox 


x X* * 


A matrix that is both upper and lower triangular is diagonal, because only the entries on the 
diagonal can be non-zero. 


3.2.7 Transpose 


Vectors do not always have to be column vectors, that is just a convention. Swapping rows 
and columns is from time to time needed. The operation that swaps rows and columns is the 
so-called transpose. The transpose of A is denoted by A’. Example: 
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So transpose takes an m X n matrix to an n X m matrix. 

A key fact about the transpose is that if the product AB makes sense then B? A? also 
makes sense, at least from the point of view of sizes. In fact, we get precisely the transpose 
of AB. That is: 


(AB)" = BT AT. 
For example, 
T 
T a : ; {0 1 2 ; ; 
4 5 6 {1 0 -2 
2 —2 3 6 


It is left to the reader to verify that computing the matrix product on the left and then 
transposing is the same as computing the matrix product on the right. 

If we have a column vector % to which we apply a matrix A and we transpose the result, 
then the row vector Z” applies to A’ from the left: 


(Az)* = #7 AP. 


Another place where transpose is useful is when we wish to apply the dot product* to 
two column vectors: 
#-J=7' Zz. 
That is the way that one often writes the dot product in software. 
We say a matrix A is symmetric if A = A’. For example, 


12 3 
2 4 5 
3.5 6 


is a symmetric matrix. Notice that a symmetric matrix is always square, that is, n x n. 
Symmetric matrices have many nice properties’, and come up quite often in applications. 
To end the section, we notice how Az can be written more succintly. Suppose 


Q11 GQ a 1 
11 G12 443 - 
A= and L= |X 
Q21 422 493 
v3 
Then 
Ly 
Agi (Ot Me | || | + Q12%2 + 413X3 
21 22 A293 Ag1L1 + A22%2Q + A93%3 


For example, 
12) 2 |) _ [Le2-4-2-(=1)/ _ |0 
3A) li) (828442 ())) ~ 13)" 
In other words, you take a row of the matrix, you multiply them by the entries in your 
vector, you add things up, and that’s the corresponding entry in the resulting vector. 


*As a side note, mathematicians write y7Z and physicists write Z7y. Shhh...don’t tell anyone, but the 
physicists are probably right on this. 
‘Although so far we have not learned enough about matrices to really appreciate them. 
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3.2.8 Exercises 


Exercise 3.2.1: Add the following matrices 


he A 2 -8 -3 
Ole pall ae b) }2 3 1]/+]3 1 0 
Os wo 6 -4 1 
Exercise 3.2.2:* Add the following matrices 
6 -2 3 —-1 -1 -3 
ew ee lr 3 af+fe 7 3 
8 -l 2 -9 4 -l 
Exercise 3.2.3: Compute 
0 3 1 5 —3 1 2 -1 
ol a) +8|2 g yl al-3fs 
Exercise 3.2.4:* Compute 
1 2 -1 3 2 -1 2 =I 
val aea[a a yal a]-3[A a 
Exercise 3.2.5: Multiply the following matrices 
—-1 2 he Decal Wes de OP! ee 
a 4 eae Dye E) |e Be Bs aT 
5 8 iio 6 anes -1 3 0 
2:55 
416 3 12 114 2 22 
c) ]}5 6 5 0 35 d) 051 1 0 
466 0 5 6 6 4 
Exercise 3.2.6:* Multiply the following matrices 
2:4 Or <s. °°3 6 6 2 
ofa ao 3 by a 9 4 lave 
3.5 8-5. =2) 2: 04 
3 4 1] {0 2 5 0 —2 -2 0 3 
6) }2=1) 0}, 20-5 2 a}s afl 
4 -1 5] {3 6 1 6 2 1 
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Exercise 3.2.7: 


a) How must the dimensions of two matrices line up in order to multiply them together? 
If they can be multiplied, what is the dimension of the product? 


b) If A is a 3 x 2 matrix and the product AB is a 3 x 4 matrix, then what are the 
dimensions of B? 


c) If A isa5 x 3 matrix, is it possible to find a matrix B so that the product AB is a 
4 x 3 matrix? What about a matrix C so that the product CA is a 4 x 3 matrix? 


Exercise 3.2.8: Assume that A is a3 x 4 matrix. 
a) What must the dimensions of B be in order for the product AB to be defined? 
b) What must the dimensions of B be in order for the product BA to be defined? 
c) What about if we want to compute ABA or BAB? 

Exercise 3.2.9: Complete Exercise 3.2.8 but with A being a 2 x 2 matrix. 


Exercise 3.2.10: Compute the inverse of the given matrices 


» [3 Tol ofa aft al 


Exercise 3.2.11:* Compute the inverse of the given matrices 


a) 2] nt ol ds 5 oft 4 


Exercise 3.2.12: Compute the inverse of the given matrices 


1 0 0 0 
3.0 =O 
—) 5 0 -1 0O 0 
2) k i P) : . : J \o 0 001 0 
0 0 0 5 
Exercise 3.2.13:* Compute the inverse of the given matrices 
-1 00 0 
Ae 9 
2 0 0 20 0 
a | A Y) ae J}o 03 0 
0 0 0 O01 


1 2 1 1 1 3 
4=[a al ®=[n a] =[o i) 
a) Compute the products AB and AC. 
b) Verify that for these matrices AB = AC, but B#C. 
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Exercise 3.2.15: Consider the matrices 
1 -l 1 1 
[a] af 
Verify that AB and BA are both equal to zero, but neither of the matrices A and B are zero. 


Exercise 3.2.16: 


a) Let A be a3 x 4 matrix. What dimension does the vector v need to be in order for 
the product Av to be defined? If this product is defined, what is the dimension of the 
product Av? 


b) Let B be a3 x 3 matrix. What dimension does the vector U need to be in order for 
the product Bv to be defined? If this product is defined, what is the dimension of the 
product Bu? 
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3.3. Elimination 


Attribution: [JL], §A.3. 


After this section, you will be able to: 


e Write a system of linear equations in matrix form, 

e Use row reduction to put a matrix into row echlon form or reduced row echelon 
form, and 

e Determine whether a system of linear equations has no solution, one solution, or 
infinitely many solutions. 


3.3.1 Linear systems of equations 


One application of matrices is to solve systems of linear equations*. Consider the following 
system of linear equations 
2X1 + 2X9 + 2x3 = 2, 
21+ %2+ 3x3 = 5, (3.2) 
v1, +4r.+ 23 = 10. 


There is a systematic procedure called elimination to solve such a system. In this 
procedure, we attempt to eliminate each variable from all but one equation. We want to end 
up with equations such as x3 = 2, where we can just read off the answer. 

We write a system of linear equations as a matrix equation: 


Az =b 
The system (3.2) is written as 
2 ee |) Way 2 
1 1 3 La) = 3) 
1 4 1] [23 10 
# b 


If we knew the inverse of A, then we would be done; we would simply solve the equation: 


#— A AZ = AW. 


Well, but that is part of the problem, we do not know how to compute the inverse for matrices 
bigger than 2 x 2. We will see later that to compute the inverse we are really solving Av = b 
for several different b. In other words, we will need to do elimination to find A~!. In addition, 
we may wish to solve Av = b even if A is not invertible, or perhaps not even square. 


*Although perhaps we have this backwards, quite often we solve a linear system of equations to find out 
something about matrices, rather than vice versa. 
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Let us return to the equations themselves and see how we can manipulate them. There 
are a few operations we can perform on the equations that do not change the solution. First, 
perhaps an operation that may seem stupid, we can swap two equations in (3.2): 


Uy+ t+ 373 =5, 
224 + 2X9 + 223 = 2, 
£1 +4¢%2+ x3 = 10. 


Clearly these new equations have the same solutions 71, 72,73. A second operation is that we 
can multiply an equation by a nonzero number. For example, we multiply the third equation 
in (3.2) by 3: 
221 =- 2X9 + 223 = 2, 
U+ Lot 3£3 = 5, 
321 + 12% + 3X3 = 30. 


Finally we can add a multiple of one equation to another equation. For example, we add 3 
times the third equation in (3.2) to the second equation: 


224 + 2X9 + 223 = 2, 
(1+3)a1+ (14+12)ao+ (34 3)23 = 5+ 30, 
ry+ Ar + a3 = 10. 


The same 21, %2, x3 should still be solutions to the new equations. These were just examples; 

we did not get any closer to the solution. We must to do these three operations in some more 

logical manner, but it turns out these three operations suffice to solve every linear equation. 
The first thing is to write the equations in a more compact manner. Given 


Az = b, 
we write down the so-called augmented matrix 


[A | 4], 
where the vertical line is just a marker for us to know where the “right-hand side” of the 
equation starts. For example, for the system (3.2) the augmented matrix is 

22 2|2 
dds eB 
14: 12 


oS 


The entire process of elimination, which we will describe, is often applied to any sort of 
matrix, not just an augmented matrix. Simply think of the matrix as the 3 x 4 matrix 


222 2 
113 5 
1441 


—_ 
jon) 
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3.3.2 Row echelon form and elementary operations 


We apply the three operations above to the matrix. We call these the elementary operations 
or elementary row operations. 


Definition 3.3.1 


The elementary row operations on a matrix are: 
(i) Swap two rows. 


(ii) Multiply a row by a nonzero number. 


(iii) Add a multiple of one row to another row. 


Note that these are the same three operations that we could do with equations to try to 
solve them earliner in this section. We run these operations until we get into a state where it 
is easy to read off the answer, or until we get into a contradiction indicating no solution. 

More specifically, we run the operations until we obtain the so-called row echelon form. 
Let us call the first (from the left) nonzero entry in each row the leading entry. A matrix is 
in row echelon form if the following conditions are satisfied: 


(i) The leading entry in any row is strictly to the right of the leading entry of the row 
above. 


(ii) Any zero rows are below all the nonzero rows. 
(iii) All leading entries are 1. 

A matrix is in reduced row echelon form if furthermore the following condition is satisfied. 
(iv) All the entries above a leading entry are zero. 


Example 3.3.1: The following matrices are in row echelon form. The leading entries are 
marked: 


Oa) 6. 2 
00 0 QJ 
0 0 


0 O 


9 3 -1 -3 2 1 

5 0 5 0. (iyi 2 
0 0 0 0 0 0 
Note that the definition applies to matrices of any size. None of the matrices above are in 
reduced row echelon form. For example, in the first matrix none of the entries above the 


second and third leading entries are zero; they are 9, 3, and 5. 
The following matrices are in reduced row echelon form. The leading entries are marked: 


(H) 


4) 3 0 8 (t) 0° 2 o (i) 0 3 0 (4) 2 0 
0 0 (1) 6 0 (1) 3 0 0 fy, = 00 0) 
0 0 0 0 0000) 0 0 0 0 0 0 0 


208 CHAPTER 3. LINEAR ALGEBRA 


The procedure we will describe to find a reduced row echelon form of a matrix is called 
Gauss—Jordan elimination. The first part of it, which obtains a row echelon form, is called 
Gaussian elimination or row reduction. For some problems, a row echelon form is sufficient, 
and it is a bit less work to only do this first part. 

To attain the row echelon form we work systematically. We go column by column, starting 
at the first column. We find topmost entry in the first column that is not zero, and we call it 
the pivot. If there is no nonzero entry we move to the next column. We swap rows to put the 
row with the pivot as the first row. We divide the first row by the pivot to make the pivot 
entry be a 1. Now look at all the rows below and subtract the correct multiple of the pivot 
row so that all the entries below the pivot become zero. 

After this procedure we forget that we had a first row (it is now fixed), and we forget 
about the column with the pivot and all the preceding zero columns. Below the pivot row, 
all the entries in these columns are just zero. Then we focus on the smaller matrix and we 
repeat the steps above. 

It is best shown by example, so let us go back to the example from the beginning of the 
section. We keep the vertical line in the matrix, even though the procedure works on any 
matrix, not just an augmented matrix. We start with the first column and we locate the 
pivot, in this case the first entry of the first column. 


We multiply the first row by 1/2. 
13-25;) 2 

LE 1 =3)| 5 
141 
We subtract the first row from the second and third row (two elementary operations). 

Bde ale 

0 0 2)4 

0 3 0/9 


We are done with the first column and the first row for now. We almost pretend the matrix 
doesn’t have the first column and the first row. 


OK, look at the second column, and notice that now the pivot is in the third row. 


el 
0 0 2/4 
0 0|9 
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We swap rows. 


1 1 11 

0 0/9 

0 0 -2)-4 
And we divide the pivot row by 3. 

1 1 11 

0 0/3 

OO. ¢2).4 


We do not need to subtract anything as everything below the pivot is already zero. We move 
on, we again start ignoring the second row and second column and focus on 


We find the pivot, then divide that row by 2: 


111/11 1 1 i1y)i1 
Oe he © SQ ts > Oe sh. 200113 
0 0 4 00 1]2 
The matrix is now in row echelon form. 

The equation corresponding to the last row is 73 = 2. We know x3 and we could substitute 
it into the first two equations to get equations for 7; and x2. Then we could do the same 
thing with x2, until we solve for all 3 variables. This procedure is called backsubstitution and 
we can achieve it via elementary operations. We start from the lowest pivot (leading entry in 
the row echelon form) and subtract the right multiple from the row above to make all the 
entries above this pivot zero. Then we move to the next pivot and so on. After we are done, 
we will have a matrix in reduced row echelon form. 

We continue our example. Subtract the last row from the first to get 


i) 


ye 8 lal 
0 1 0; 3 
OS 0 ee De 


The entry above the pivot in the second row is already zero. So we move onto the next pivot, 
the one in the second row. We subtract this row from the top row to get 


1 0 
Ors 1 
0 0 


The matrix is in reduced row echelon form. 
If we now write down the equations for x71, 22,73, we find 


t= —4, v2 >= 3, v3 = 2. 
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In other words, we have solved the system. 
Example 3.3.2: Solve the following system of equations using row reduction: 


—%+%,+3%73=7 


—3271 TT 3S —5 
—29%, —% = —4 


Solution: In order to solve this problem, we need to set up the augmented matrix for this 
system, which is 


=F od abe). 
3.000 d= 6 
—2 -1 0|-4 


To carry out the process, we need to get a 1 in the top left corner, then work from there. 
We multiply the first row by -1 to get 


Lie aks S| ak 
sare Ab) 45 
—2 -1 0 |-4 


Next, we want to use row 1 to cancel out the —3 and —2 in column 1. To do this, we add 
three copies of row 1 to row 2, and two copies of row 1 to row 3 to get the augmented matrix 


a) eo 
0 -3 —8 | —26 
0 —3 —6|-—18 


Normally, the next step would be to divide the second row by —3 in order to put a 1 in 
that pivot spot. However, since both the second and third rows have a —3 in the second 
column, we can combine these two rows directly without dividing by —3 first. We subtract 
row 2 from row 3 to get 

1 -1 -3] -7 
0 -—3 -—8 | —26 


0 0 2 8 
and we can now use this to solve the system. The bottom row says that 273 = 8, so 
that +3 = 4. The second row says that —3x2 — 8%3 = —26, since 73 = 4, we have that 
—32o = —26 + 32 = 6, so rg = —2. Finally, the first row of the augmented matrix says 
that 7; — 2 — 3x3 = —7. Plugging in our values for x2 and 23 gives 7} = —7 —-24+12=3. 
Therefore, the solution is 
y= 3 Lo = —2 3 =A. all 


3.3.3 Non-unique solutions and inconsistent systems 


It is possible that the solution of a linear system of equations is not unique, or that no 
solution exists. Suppose for a moment that the row echelon form we found was 


1 2 3/4 
0 1/3 
0 0} 1 
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Then the last row gives the equation 02, + O22 + Ox3 = 1, or 0 = 1. That is impossible and 
the equations are inconsistent. There is no solution to Ax = b. 
On the other hand, if we find a row echelon form 


ie 
0 0 
0 0 


oF Ww 


4 

3 |, 

0 

then there is no issue with finding solutions. In fact, we will find way too many. Let us 


continue with backsubstitution (subtracting 3 times the third row from the first) to find the 
reduced row echelon form and let’s mark the pivots. 


(1) 2 o|-5 
0 0 Gj) 3 
0 0 0} 0 


The last row is all zeros; it just says 0 = 0 and we ignore it. The two remaining equations are 
y+ 229 = —5, 03 = 3. 


Let us solve for the variables that corresponded to the pivots, that is x, and x3 as there was 
a pivot in the first column and in the third column: 


v= —225 = 5, 


r3 = 3. 


The variable x2 can be anything you wish and we still get a solution. The 72 is called a free 
variable. There are infinitely many solutions, one for every choice of x2. For example, if we 


pick x2 = 0, then x; = —5, and x3 = 3 give a solution. But we also get a solution by picking 
say X2 = 1, in which case x; = —9 and 23 = 3, or by picking x2 = —5 in which case x1 = 5 
and 73 =3. 


The general idea is that if any row has all zeros in the columns corresponding to the 
variables, but a nonzero entry in the column corresponding to the right-hand side b, then the 
system is inconsistent and has no solutions. In other words, the system is inconsistent if you 
find a pivot on the right side of the vertical line drawn in the augmented matrix. Otherwise, 
the system is consistent, and at least one solution exists. 

If the system is consistent: 


(i) If every column corresponding to a variable has a pivot element, then the solution is 
unique. 


(ii) If there are columns corresponding to variables with no pivot, then those are free 
variables that can be chosen arbitrarily, and there are infinitely many solutions. 


Another way to interpret this idea of free variables is that at the beginning, before you look 
at the system of equations, all of the variables can be anything, and there are no constraints 
on them. The equations then give us constraints on these variables, because they give us 
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rules that the variables must satisfy. When we have a row of the augmented matrix that 
becomes all zeros, it means that the equation that was there is redundant and doesn’t add 
any constraints to the equations. This may result in an underdetermined system, which will 
likely have free variables. 


Example 3.3.3: Solve the following two systems of equations, or determine that no solution 
exists, using row reduction: 


4 Os Bq — 343 = —3 v1 Lq — 323 = —3 
—%X1 — 249 + 403 = 6 —%1 — 2%. + 403 = 6 
1+ 9X9 — 5x3 = —9 1, + 5X2 — 573 = 1 


Solution: For the first of these systems, we will set up the augmented matrix and proceed 
through the process like normal. The augmented matrix is 


1 -1 -3)]-8 
—1 -2 4] 6 
1 5 -5]|-9 
Since we already have a 1 in the top-left corner of this matrix, we can use it to cancel the 


entries in the rest of column 1. We add one copy of row 1 to row 2, and subtract row 1 from 
row 3 to get the next augmented form matrix as 


Po) 33 
0-3 1) 3 
0 6 —2/-—6 


Looking at the matrix here, we see that row 3 is —2 times row 2. Therefore, if we add two 
copies of row 2 to row 3, we get the augmented matrix 


dh Sat e8 
0-3 1) 3 
0 0 O07; 0 


Therefore, we have a situation where there are only two pivot columns, and the last row is 
all zeros. Since there are three variables and column 3 is not a pivot column, we can take x3 
as a free variable. If we do that, the second equation tells us that —322 + x3 = 3, or, since 
we are taking x3 as a free variable, we can write v2 = —1 + 1/373. We can then take the first 
equation, which says that 71 — x2 — 3x3 = 3 or, by rearranging 


1 10 
%=34+%.+ 393 =34 (-1 ) Ree ag as 


This means that for any value of t, our solution is determined by 


10 
er 
ieee 
sf ae ar 


r3=t 
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The use of t here is just to separate it from the variable x3. For example, we could pick t = 3, 
in which case we would get x, = 12, x2 = 0, x3 = 3. 
For the second version of the problem, we again set up the augmented matrix 


1 -1 -—3)]-3 
—-1 -2 41] 6 
1 5 -5)] 1 
Since the left side matrix part is the same as the previous version, the process of row reducing 


the matrix is identical to what was done previously. When we carry out this process we get 
the augmented matrix 


1 -1 —3)-8 

0-3 1) 3 

0 0 O | 10 
In this case, we see that the last row corresponds to the equation 0 = 10 so these equations 
are inconsistent and do not have a solution. | 


The point of the above example is to illustrate the fact that whether or not a system 
is inconsistent or has free variables in the solution depends on the right-hand side of the 
equation, even if the left-hand side has the same coefficients. We’ll see more about why this 
is in § 3.5. 


When 6 = 0, we have a so-called homogeneous matrix equation 
Az =0. 


There is no need to write an augmented matrix in this case. As the elementary operations do 
not do anything to a zero column, it always stays a zero column. Moreover, AZ = 0 always 
has at least one solution, namely ¢ = 0. Such a system is always consistent. It may have 
other solutions: If you find any free variables, then you get infinitely many solutions. 

The set of solutions of Az = 0 comes up quite often so people give it a name. It is called 
the nullspace or the kernel of A. One place where the kernel comes up is invertibility of a 
square matrix A. If the kernel of A contains a nonzero vector, then it contains infinitely many 
vectors (there was a free variable). But then it is impossible to invert 0, since infinitely many 
vectors go to 0, so there is no unique vector that A takes to 0. So if the kernel is nontrivial, 
that is, if there are any nonzero vectors, in other words, if there are any free variables, or 
in yet other words, if the row echelon form of A has columns without pivots, then A is not 
invertible. We will return to this idea later. 


3.3.4 Exercises 


Exercise 3.3.1: Compute the reduced row echelon form for the following matrices: 


Mori» [6 SI 95S) afta” 


i 1 

9302 TB e 6 6 5 73 @ 1 
e) 18 6 3 6 16 0 0 =1) # lo 2 2 h) |6 6 -3 3 
7979 244 8 6 5 6 62-3 5 
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Exercise 3.3.2:* Compute the reduced row echelon form for the following matrices: 


1-3 1 
101 152 i 
eee o[Z4)  of4 4 
0 10 3 4 2 2 a eee 
2-2: oD 26 4 3 

0000 i a 
e) |1 -2 4 -1] f) |6 0 -3 0 D5 aoa Doe 
0 3 1-2 A, Dosied 


Exercise 3.3.3: Solve (find all solutions), or show no solution exists 


a eee v1 + 52+ 3273 =7 

a) b) 8x1 + 7x2 + 823 = 8 
a a 4x, + 8%. + 673 = 4 
47, + 8x2 + 243 = 3 a+2y+3z=4 

c) —% — 2%. +3273 =1 d) 2r— y+3z=1 
4x1 + 82x =2 3x y+6z=6 


Exercise 3.3.4:* Solve (find all solutions), or show no solution exists 


ox + 6y +5z=7 


4a, + 329 = —1 
a) Pantna b) 62 + 8y + 6z = —1 
OLy an 62x =4 
5x2 + 2y + 5z = 2 
a b c=-l —2271 + 272 + 843 = 6 
c) a+5b+6c=-1 d) tot x3=2 
—2a+5b+6c=8 11+ 4x 13 = 


Exercise 3.3.5:* Solve the system of equations 


—4ro + 43+ 2x, = 16 
2X1 + 2x9 — 403 — 344 = 1 
1 +%2+ 273+ 344 =6 
2x, — 243+ 4x4 = 24 


or determine that no solution exists. 
Exercise 3.3.6:* Solve the system of equations 


3X2 + 343 + 2%, = 4 
4a, + 4x2 + 2x3 — 4244 = —26 
U1 — 3% — 2473 4+ 2%, = 1 
321 + 3% + 3%3 — 44 = —14 


or determine that no solution exists. 
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Exercise 3.3.7:* Solve the system of equations 


24, + %2—2%3+4r, = 11 

v1, +40. —423—-24, = —7 
—27, — 3%). +273 +244 = 11 
34, +43 +47, = 3 


or determine that no solution exists. 
Exercise 3.3.8:* Solve the system of equations 


v1 — £3 — 4%, = —3 
t1+%2+%4=0 
v1 + 3% + 323 — 4x44 = —28 
6x1 + 3x9 — 423 + 6244 = 25 


or determine that no solution exists. 


Exercise 3.3.9:* Assume that you are solving a three component linear system of equations 
via row reduction of an augmented matrix and reach the matrix 


1. O.-8:)'4 
0 0'.10)'3 
0 0 Of 1 
What does this mean about the solution to this system of equations? 


Exercise 3.3.10:* Assume that you are solving a three component linear system of equations 
via row reduction of an augmented matrix and reach the matrix 


1 1 3/6 
0 1 2/4 
0 0 0/0 
What does this mean about the solution to this system of equations? 


Exercise 3.3.11: Assume that you are solving a four component linear system of equations 
via row reduction of an augmented matrix and reach the matrix 


What is the next step in reducing this matrix? Carry out the rest of this problem to solve 
the corresponding system of equations. 
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Exercise 3.3.12:* Assume that someone else has provided you the solution to an augmented 

matrix reduction for solving a system of equations given below 
112 4 112 4 1 1 2 
013 5} > }0 13 5] > j0 01 
124 -1 012 -5 000 -9 

Is this work correct? If so, what does this say about the solution(s) to the system? If not, 


correct the work to solve the system. 
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3.4 Linear independence, rank, and dimension 


Attribution: [JL], §A.4. 


After this section, you will be able to: 


e Determine if a set of vectors is linearly independent, 

e Compute the rank of a matrix, 

e Find a maximal linearly independent subset of a set of vectors, and 
e Compute a basis of a subspace and the dimension of that subspace. 


3.4.1 Linear independence and rank 


As we saw in the § 3.3, it is possible to have a set of equations that is redundant; that is, 
at least one of the equations does not give us any more information or constraints on the 
variables. In a lot of cases, this either led to inconsistent systems or free variables. We would 
like to have a better way to talk about this idea, both in terms of systems of equations and 
matrices in general. The concept we want is that of linear independence. The same concept 
is useful for differential equations, for example in Chapter 2. 


Definition 3.4.1 


Given row or column vectors {7}, Yo,.--, Yn, a linear combination is an expression of the 
form 


Oi 20a + On Un, 


where a1, Q2,..., Qn» are all scalars. 


For example, 37) + Yo — 5y3 is a linear combination of y}, Yo, and 73. 
We have seen linear combinations before. The expression 


AZ 
is a linear combination of the columns of A, while 
e A= (Aa) 
is a linear combination of the rows of A. 
The way linear combinations come up in our study of differential equations is similar to 
the following computation. Suppose that 7), 2, ..., Z, are solutions to Az, = 0, A®2 = 0, 


..., AZ, = 0. Then the linear combination 


Y = 12, + Ag +--+ + AnXn 
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is a solution to Ag = 0: 
Ay = A(ai 21 + Q2%o + +--+ Ontn) = 
= 0, AT) + aQAT. +--+ + Qa,AEn = a0 + a20 + seep a,0 = 0. 


We have seen this computation before in the sense of solutions to homogeneous second 
order equations. We used L[y] to represent a second order linear differential equation, and 
showed that if we knew that functions y, and y2 solved 


Lly:] = 0 Lly2| = 0 
then Liciy1 + c2y2| = 0 for any constants cy and cy. We did this by showing that 
Lley1 + e2y2] = a Ll yi] + CoL [yo] 


which mirrors the expression computed above. 
Our original question was about when equations are redundant. That is answered by the 
following definition. 


Definition 3.4.2 


We say the vectors 1, Y2, ---,; Yn are linearly independent if the only way to pick 
Q1, Q2,..., Qn to satisfy 


Q421 7B vee AnkLn = 0 


is a] = Q2 =--- =a, = 0. Otherwise, we say the vectors are linearly dependent. 


If the equations (or their coefficients, as we will see later) are linearly dependent, then 
they are redundant equations, and not all of them are necessary to define the same solution 
to the equation. If they are linearly independent, then they are all required. 


Example 3.4.1: The vectors [4] and [?] are linearly independent. 


ob +l = ato] = [a 


So a, = 0, and then it is clear that a2 = 0 as well. In other words, the vectors are linearly 
independent. | 


Solution: Let’s try: 


If a set of vectors is linearly dependent, that is, we have an expression of the form 


0421 + AgFo +--+ +AnXn = 0 


with some of the a;’s are nonzero, then we can solve for one vector in terms of the others. 
Suppose a, # 0. Since a,%1 + Q2%q +--+ + AnFp = 0, then 
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For example, 


1 1 1 0 
2/2) —4])1/+2]0] = 1/0], 
3 i —1 0 
and so 
1 1 1 
2) = 2,1) (0 
3 1 —1l 


You may have noticed that solving for those a,’s is just solving linear equations, and so 
you may not be surprised that to check if a set of vectors is linearly independent we use row 
reduction. 

Given a set of vectors, we may not be interested in just finding if they are linearly 
independent or not, we may be interested in finding a linearly independent subset. Or 
perhaps we may want to find some other vectors that give the same linear combinations and 
are linearly independent. The way to figure this out is to form a matrix out of our vectors. If 
we have row vectors we consider them as rows of a matrix. If we have column vectors we 
consider them columns of a matrix. 


Definition 3.4.3 


Given a matrix A, the maximal number of linearly independent rows is called the rank 
of A, and we write “rank A” for the rank. 


For example, 
1 1 1 
rank |) 2. 2: 2) 1, 
—1 -1 -1l 
The second and third row are multiples of the first one. We cannot choose more than one 
row and still have a linearly independent set. But what is 


1 2 8 
rank |4 5 6 = ? 
7 8 9 


That seems to be a tougher question to answer. The first two rows are linearly independent, 
so the rank is at least two. If we would set up the equations for the a1, a2, and a3, we would 
find a system with infinitely many solutions. One solution is 


[1 2 3]-2/4 5 6]+[7 8 9]/=(0 0 O}. 


So the set of all three rows is linearly dependent, the rank cannot be 3. Therefore the rank is 
2. 
But how can we do this in a more systematic way? We find the row echelon form! 


1 2 3 12 3 
Row echelon form of 45 6 is 0 1 2 
7 8 9 00 0 
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The elementary row operations do not change the set of linear combinations of the rows 
(that was one of the main reasons for defining them as they were). In other words, the span 
of the rows of the A is the same as the span of the rows of the row echelon form of A. In 
particular, the number of linearly independent rows is the same. And in the row echelon 
form, all nonzero rows are linearly independent. This is not hard to see. Consider the two 
nonzero rows in the example above. Suppose we tried to solve for the a; and a, in 


a; {1 2 3)+a2[0 1 2]=[0 0 0]. 


Since the first column of the row echelon matrix has zeros except in the first row means 
that a, = 0. For the same reason, @2 is zero. We only have two nonzero rows, and they are 
linearly independent, so the rank of the matrix is 2. This also tells us that if we were trying 
to solve the system of equations 


%1+2%2+ 373 =a 
4x, + 529 + 623 = D 
7a, + 8%. + 943 =Cc 


we would get that one row of the reduced augmented matrix has all zeros on the left side, 
and so this system either has a free variable or is inconsistent, because only two equations 
here are relevant. We will see more examples of the rank of a matrix once we have more 
terminology to talk about it. 


3.4.2 Subspaces and span 


Now, let’s consider a different scenario. Assume that we find two vectors that solve Az = 0. 
What other vectors also solve this equation? In our discussion of linear combinations, we saw 
that if 7, and £» solve A¥ = 0, then so does A(a;#, + Q2%2) for any constants a; and apg. 
Thus, all linear combinations will also solve the equation. This leads to the definition of the 
span of a set of vectors. 


Definition 3.4.4 


The set of all linear combinations of a set of vectors is called their span. 


span{ £1, Easton tnt = {Set of all linear combinations of #1, %2,... Bee 


Thus, if two vectors solve a homogeneous equation, so does everything in the span of 
those two vectors. The span of a collection of vectors is an example of a subspace, which 
is a common object in linear algebra. We say that a set S of vectors in R” is a subspace if 
whenever # and y are members of S and a is a scalar, then 


Z£+Y, and axt 
are also members of S. That is, we can add and multiply by scalars and we still land in S. 
So every linear combination of vectors of S is still in S. That is really what a subspace is. It 
is a subset where we can take linear combinations and still end up being in the subset. 
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Example 3.4.2: If we let S = R”, then this S' is a subspace of R”. Adding any two vectors 
in R” gets a vector in R”, and so does multiplying by scalars. 

The set S’ = {0}, that is, the set of the zero vector by itself, is also a subspace of R”. 
There is only one vector in this subspace, so we only need to check for that one vector, and 
everything checks out: 0+ 0 =0 and a0 = 0. 

The set S” of all the vectors of the form (a, a) for any real number a, such as (1,1), (3,3), 
or (—0.5, —0.5) is a subspace of R*. Adding two such vectors, say (1, 1) + (3,3) = (4,4) again 
gets a vector of the same form, and so does multiplying by a scalar, say 8(1, 1) = (8,8). 


We can apply these ideas to the vectors that live inside a matrix. The span of the rows of 
a matrix A is called the row space. The row space of A and the row space of the row echelon 
form of A are the same, because reducing the matrix A to its row echelon form involves 
taking linear combinations, which will preserve the span. In the example, 


1.2 3 
row space of |4 5 6 = span { [1 2 3] , [4 5 6] , [7 8 9| } 
too 


= span { [1 2 3] , [0 i OL 


Similarly to row space, the span of columns is called the column space. 


1 2.3 1 2 3 
column space of |4 5 6] =span ¢ [4], [5], |6 
7 8 9 7 8 9 


So it may also be good to find the number of linearly independent columns of A. One 
way to do that is to find the number of linearly independent rows of A’. It is a tremendously 
useful fact that the number of linearly independent columns is always the same as the number 
of linearly independent rows: 


Theorem 3.4.1 


rank A = rank AT 


In particular, to find a set of linearly independent columns we need to look at where the 
pivots were. If you recall above, when solving AZ = 0 the key was finding the pivots, any 
non-pivot columns corresponded to free variables. That means we can solve for the non-pivot 
columns in terms of the pivot columns. Let’s see an example. 


Example 3.4.3: Find the linearly independent columns of the matrix 


123 4 
24 5 6 
a2. 8 


Solution: We find a pivot and reduce the rows below: 


[i] 23 4 [ij 2 3 4 (ye <A 


2 4.8 654 10 0 =) =2) 4/0 0 =1. 22 
3.6 7 8 3.6 7 8 0 0 -2 -4 
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We find the next pivot, make it one, and rinse and repeat: 


i (i) so 2 A (1) 2 3 4 
0 0 (-1) -2)>}0 0 @ 2}/-]0 0 @) 2 
0 0 -2 -4 0 0 -2 -4 000 0 


The final matrix is the row echelon form of the matrix. Consider the pivots that we marked. 
The pivot columns are the first and the third column. All other columns correspond to free 
variables when solving Az = 0, so all other columns can be solved in terms of the first and 
the third column. In other words 


1234 1 2 3 4 1 3 
column space of |2 4 5 6] =span< [2], |4], 1/5], 16 =span <2), | 
3 6 7 8 3 6 7 8 3 7 


We could perhaps use another pair of columns to get the same span, but the first and 
the third are guaranteed to work because they are pivot columns. The discussion above 
could be expanded into a proof of the theorem if we wanted. As each nonzero row in the row 
echelon form contains a pivot, then the rank is the number of pivots, which is the same as 
the maximal number of linearly independent columns. 

In the previous example, this means that only the first and third colums are “important” 
in the sense of generating the full column space as a span. We would like to have a way to 
talk about what these first and third columns do. 


Definition 3.4.5 


Let S be a subspace of a vector space. The set {Uj, v2, ...,U,} is a spanning set for the 
subspace S if each of these vectors are in S and the span of {Uj, U2, ...,¥,} is equal to S. 


In the context of the previous example, for the matrix 


123 4 
A=|2 4 5 6 
3.67 8 
we know that 
1234 1 2 3 4 1 3 
column space of |2 4 5 6] =span¢ [2], |4], [5], |6 =span< |2], [5 
3 6 7 8 3 6 c 8 3 ¢ 
This means that both 
1 2 S 4 1 3 
2) 14 oc 2 and Oy yo 
3 6 7 8 3 ij 


are spanning sets for this column space. 
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The idea also works in reverse. Suppose we have a bunch of column vectors and we just 
need to find a linearly independent set. For example, suppose we started with the vectors 


St 
I 


1 2 3 4 
2 5 U2 = 4 5 v3 = 5 5 i= 6 
3 6 v4 8 


These vectors are not linearly independent as we saw above. In particular, the span v) and v3 
is the same as the span of all four of the vectors. So V2 and v4 can both be written as linear 
combinations of v,; and v3. A common thing that comes up in practice is that one gets a set 
of vectors whose span is the set of solutions of some problem. But perhaps we get way too 
many vectors, we want to simplify. For example above, all vectors in the span of 1, U2, U3, U4 
can be written aU , + Qt + a303 + a4v4 for some numbers aj, 2,3, Q4. But it is also true 
that every such vector can be written as av, + bv3 for two numbers a and b. And one has to 
admit, that looks much simpler. Moreover, these numbers a and 6 are unique. More on that 
later in this section. 

To find this linearly independent set we simply take our vectors and form the matrix 
[v, Vp U3 U4), that is, the matrix 


wn r 
a Ww 
“JOT Ww 


4 
6 
8 


jon 


We crank up the row-reduction machine, feed this matrix into it, and find the pivot columns 


and pick those. In this case, v; and v3. 


3.4.3 Basis and dimension 


At this point, we have talked about subspaces, and two other properties of sets of vectors: 
linear independence and being a spanning set for a subspace. In some sense, these two 
properties are in opposition to each other. If I add more vectors to a set, I am more likely to 
become a spanning set (because I have more options for adding to get other vectors), but 
less likely to be independent (because there are more possibilities for a linear combination to 
be zero). Similarly, the reverse is true; removing vectors means the set is more likely to be 
linearly independent, but less likely to span a given subspace. The question then becomes if 
there is a sweet spot where both things are true, and that leads to the definition of a basis. 


Definition 3.4.6 


If S is a subspace and we can find k linearly independent vectors in S 


Uy UDonea eo) Uk 


such that every other vector in S' is a linear combination of Uj, Vo,...,U,, then the set 
{U1, Uo,..., Ux} is called a basis of S. In other words, S is the span of {tv}, Uo,...,U,}. 
We say that S has dimension k, and we write 


dim S = k. 
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The next theorem illustrates the main properties and classification of a basis of a vector 
space. 


Theorem 3.4.2 


If S C R” is a subspace and S is not the trivial subspace {0}, then there exists a unique 
positive integer k (the dimension) and a (not unique) basis {dv}, v,...,0,}, such that 
every w in S can be uniquely represented by 


i — QV} + AV ap 289 + Qp,Up, 


for some scalars @1, Qo, ..., Gp. 


We should reiterate that while k is unique (a subspace cannot have two different dimen- 
sions), the set of basis vectors is not at all unique. There are lots of different bases for any 
given subspace. Finding just the right basis for a subspace is a large part of what one does in 
linear algebra. In fact, that is what we spend a lot of time on in linear differential equations, 
although at first glance it may not seem like that is what we are doing. 


Example 3.4.4: The standard basis 


€1, €2,--+,€n, 


is a basis of IR” (hence the name). So as expected 


dim R” = n. 


On the other hand the subspace {0} is of dimension 0. 

The subspace S” from a previous example, that is, the set of vectors (a, a) is of dimension 1. 
One possible basis is simply {(1,1)}, the single vector (1,1): every vector in S” can be 
represented by a(1,1) = (a,a). Similarly another possible basis would be {(—1, —1)}. Then 
the vector (a,a) would be represented as (—a)(—1,—1). In this case, the subspace S” has 
many different bases, two of which are {(1,1)} and {(—1, —1)}, and the vector (a,a) has a 
different representation (different constant) for the different bases. 


Row and column spaces of a matrix are also examples of subspaces, as they are given as 
the span of vectors. We can use what we know about rank, row spaces, and column spaces 
from the previous section to find a basis. 


Example 3.4.5: Earlier, we considered the matrix 
12a 4 
A= |2 4 5-6 
3 67 8 
Using row reduction to find the pivot columns, we found 


1234 
column space of A} |2 4 5 6 = span < [2] ,|5 
a6: 7 3 
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What we did was we found the basis of the column space. The basis has two elements, and 
so the column space of A is two dimensional. Notice that the rank of A is two. 


We would have followed the same procedure if we wanted to find the basis of the subspace 
X spanned by 


3 6 7 8 


We would have simply formed the matrix A with these vectors as columns and repeated the 
computation above. The subspace X is then the column space of A. 


Example 3.4.6: Consider the matrix 
Lo a 
L=|0 0 10.4 
0001 5 


Conveniently, the matrix is in reduced row echelon form. The matrix is of rank 3. The 
column space is the span of the pivot columns, because the pivot columns always form a 
basis for the column space of a matrix. It is the 3-dimensional space 


1 0 0 
column space of L = span< |O] ,}|1], |0 = R?. 
0 0 1 


The row space is the 3-dimensional space 


row space of L=span{{1 2 0 0 3],/0 0 1 0 4],[0 0 0 1 5}. 


As these vectors have 5 components, we think of the row space of L as a subspace of R°. 


The way the dimensions worked out in the examples is not an accident. Since the number 
of vectors that we needed to take was always the same as the number of pivots, and the 
number of pivots is the rank, we get the following result. 


Theorem 3.4.3 (Rank) 


The dimension of the column space and the dimension of the row space of a matrix A 
are both equal to the rank of A. 


3.4.4 Exercises 


Exercise 3.4.1: Compute the rank of the given matrices 


6 35 — oi i @ 3 
a) {1 4 1 b) |3. 0 6 dial 2 <3 
7 7 6 24 5 2 4 6 


Exercise 3.4.2:* Compute the rank of the given matrices 
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7 —-l 6 i ae Ee | 0 3 -l 
a) |7 7 7 B) jf d c) |}6 3 1 
fr © 2 2 2 2 4 7 -1 


Exercise 3.4.3: For the matrices in Exercise 3.4.1, find a linearly independent set of row 
vectors that span the row space (they don’t need to be rows of the matrix). 


Exercise 3.4.4: For the matrices in Exercise 3.4.1, find a linearly independent set of columns 
that span the column space. That is, find the pivot columns of the matrices. 


Exercise 3.4.5:* For the matrices in Exercise 3.4.2, find a linearly independent set of row 
vectors that span the row space (they don’t need to be rows of the matrix). 


Exercise 3.4.6:* For the matrices in Exercise 3.4.2, find a linearly independent set of 
columns that span the column space. That is, find the pivot columns of the matrices. 


Exercise 3.4.7: Compute the rank of the matrix 
10 -—2 11 -7 
—5 —2 -5 5 
1 OO -4 -4 
1 2 2 -1 


Exercise 3.4.8: Compute the rank of the matrix 


4 -2 0 -4 
3 —-5 2 0 
1 =—2 0 1 
-l 1 3 -8 


Exercise 3.4.9: Find a linearly independent subset of the following vectors that has the 
same span. 


=, Z —2 =| 
1], =) 5 4], 3 
2 —4 1 2 


Exercise 3.4.10:* Find a linearly independent subset of the following vectors that has the 
same span. 


0 3 0 —3 
O} s Li 3 |, 2 
0 —5 = 4 


Exercise 3.4.11: For the following sets of vectors, determine if the set is linearly independent. 
Then find a basis for the subspace spanned by the vectors, and find the dimension of the 
subspace. 


1 —] 1 0 0 —4 2 2 
ey a ee b) jol, |1{, |-2 a (23h, Ialy. Wo 
1 —1 3) 0 0 9) 3 2 
1 0 —1l 3 2 —5 
d) 3 ’ 2 ; =] e) H ’ H ; -] f) 1 ’ d ’ =o) 
0 2 2 3 —4 —2 
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Exercise 3.4.12:* For the following sets of vectors, determine if the set is linearly indepen- 
dent. Then find a basis for the subspace spanned by the vectors, and find the dimension of 
the subspace. 


1 1 1 2 1 3) 3) -1 

a) H ; H ds a a Fi 

1 2 2 1 3) —4 
2 2 4 1 2 0 
all, fa}, Jal off]. [el [ol tof, ol, fa 
4 3 —3 0 0 2 


Exercise 3.4.13: Suppose that X is the set of all the vectors of R® whose third component 
is zero. Is X a subspace? And if so, find a basis and the dimension. 


Exercise 3.4.14:* Consider a set of 3 component vectors. 
a) How can it be shown if these vectors are linearly independent? 


b) Can a set of 4 of these 3 component vectors be linearly independent? Explain your 
answer. 


c) Can a set of 2 of these 3 component vectors be linearly independent? Explain. 


d) How would it be shown if these vectors make up a spanning set for all 3 component 
vectors? 


e) Can 4 vectors be a spanning set? Explain. 
f) Can 2 vectors be a spanning set? Explain. 


Exercise 3.4.15:* Consider the vectors 
4 3 1 
Uv, = | 2 U_q = {5 —1 
-1 1 —1 
Let A be the matrix with these vectors as columns and 6 the vector [1 0 O). 
a) Compute the rank of A to determine how many of these vectors are linearly independent. 
b) Determine if b is in the span of the given vectors by using row reduction to try to solve 
Az = b. 
c) Look at the columns of the row-reduced form of A. Is b in the span of those vectors? 


d) What do these last two parts tell you about the span of the columns of a matrix, and 
the span of the columns of the row-reduced matrix? 


e) Now, build a matrix D with these vectors as rows. Row-reduce this matrix to get a 
matrix Do. 


f) Is b in the span of the rows of D2? You can’t check this in using the matrix form; 
instead, just brute force it based on the form of Dz. What does this potentially say 
about the span of the rows of D and the rows of D2? 
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Exercise 3.4.16: Complete Exercise 3.4.15 with 


1 —6 —13 1 
, 10 4. Pd 3 7 aN 7_ |0 
Be Vesti, = ag a a ae ~ [0 

0 1 1 0 
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3.5 Determinant 


Attribution: [JL], §A.6. 


After this section, you will be able to: 


e Compute the determinant of a 2 x 2 matrix, 


e Use cofactor expansion to compute the determinant of larger matrices, and 
e Use the determinant to make statements about invertibility or rank of a matrix, 
and linear independence of the columns of that matrix. 


For square matrices we define a useful quantity called the determinant. We define the 
determinant of a 1 x 1 matrix as the value of its only entry 


det ( [a] ) wa. 


det ((" “1) - ad — be. 


Before defining the determinant for larger matrices, we note the meaning of the determinant. 
Annxn matrix gives a mapping of the n-dimensional euclidean space R” to itself. In particular, 
a 2 x 2 matrix A is a mapping of the plane to itself. The determinant of A is the factor by 
which the area of objects changes. If we take the unit square (square of side 1) in the plane, 
then A takes the square to a parallelogram of area |det(A)|. The sign of det(A) denotes a 
change of orientation (negative if the axes get flipped). For example, let 


1 1 
a-[4 I) 
Then det(A) = 1+1 = 2. Let us see where A sends the unit square with vertices (0,0), (1,0), 
(0,1), and (1,1). The point (0,0) gets sent to (0,0). 
1 1) f1) _ [2 
—1 1} }1} Jol’ 


1 uf fa 1 1) fo) fi 
foal fel=[)- [al = 
The image of the square is another square with vertices (0,0), (1,—1), (1,1), and (2,0). 
The image square has a side of length V2 and is therefore of area 2. See Figure 3.5 on the 
following page. 
In general the image of a square is going to be a parallelogram. In high school geometry, 
you may have seen a formula for computing the area of a parallelogram with vertices (0,0), 
(a,c), (b,d) and (a+6b,c+d). The area is 


det ([" i) | = lad ~ et 


For a 2 x 2 matrix we define 
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Figure 3.5: Image of the unit quare via the mapping A. 


The vertical lines above mean absolute value. The matrix [¢ 5] carries the unit square to the 


given parallelogram. 


There are a number of ways to define the determinant for an n x n matrix. Let us use 
the so-called cofactor expansion. We define Aj; as the matrix A with the i" row and the j™ 
column deleted. For example, 


4 6 


1 3 12 
If A= |4 6 5 then Aj = and Ao = : 
7 9 (a) 7 8 


Co Ot bd 


We now define the determinant recursively 
det(A) = S79 (-1)'M ay; det(Ay)), 
j=1 
or in other words 


+din det(A;,) if n is odd, 


det(A) = ay; det(A1,) — ay. det(Ay2) + det(A,3) —--- 
4) ee 12 det(Arz) + ais det(Ars) ene if nm even. 


For a 3 x 3 matrix, we get det(A) = ay; det(Ai1) — aig det(Ai2) + a3 det(Ai3). For example, 


s])=1-00 ([§ f]) -2-ae (ff 9) +5. ([F §) 


= 1(5-9-6-8) -2(4-9-6-7) +3(4-8-5-7) =0. 


det 


NR 
Cc ot bd 


It turns out that we did not have to necessarily use the first row. That is for any 2, 


n 


det(A) = S¢ (-1)aj; det (Ajj). 


1 


It is sometimes useful to use a row other than the first. In the following example it is more 
convenient to expand along the second row. Notice that for the second row we are starting 
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with a negative sign. 


1 2 3 
det 05 0 = -0-aet([5 3] ) + 8-aet (|p 3) ~o-dee (|p |) 
7 8 9 
=0+5(1-9-3-7)+0=—60. 


Let us check if it is really the same as expanding along the first row, 


1 2 3 
det 0 5 0 = 1-det a0 —2- det ee + 3- det O29 
789 8 9 7 9 7 8 


= 1(5-9—0-8) —2(0-9—0-7) +3(0-8—5-7) = —60. 


In computing the determinant, we alternately add and subtract the determinants of the 
submatrices A;; multiplied by a;; for a fixed i and all 7. The numbers (—1)’*’ det(Aj;) are 
called cofactors of the matrix. And that is why this method of computing the determinant is 
called the cofactor expansion. 

Similarly we do not need to expand along a row, we can expand along a column. For any 


n 


det(A) = S¢ (1) aj det (Ajj). 


i=1 
A related fact is that 
det(A) = det(A’). 


Recall that a matrix is upper triangular if all elements below the main diagonal are 0. For 
example, 


1 
0 
0 


So ov bw 
OD Ww 


is upper triangular. Similarly a lower triangular matrix is one where everything above the 
diagonal is zero. For example, 


1 0 0 

AY iG 

7 8 9 
The determinant for triangular matrices is very simple to compute. Consider the lower 
triangular matrix. If we expand along the first row, we find that the determinant is 1 times 


the determinant of the lower triangular matrix [39]. So the deteriminant is just the product 
of the diagonal entries: 


De £0 0) 
det | [4 5 0 =1-5-9=465. 
tT 3:9 
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Similarly for upper triangular matrices 


det =1-5-9=45. 


oc Fe 
oS ovr 
ODwWw 


In general, if A is triangular, then 
det (A) = 411922°°:*Ann- 


If A is diagonal, then it is also triangular (upper and lower), so same formula applies. For 
example, 


In particular, the identity matrix J is diagonal, and the diagonal entries are all 1. Thus, 
det(I) = 1. 


Another way that we can compute determinants is by using row reduction. Since the 
row echelon form is a diagonal matrix, this will make it easy to compute the determinant 
using the product of the diagonal entries. However, we need to know how the determinant is 
affected by elementary row operations. 


Theorem 3.5.1 (Properties of the Determinant) 
Let A be a square n X n matrix. 


1. If B obtained from A by interchanging two rows (or two columns) of A, then 
det(B) = — det(A). 


2. If B is obtained from A by multiplying a row of column by the number r, then 
det(B) = r det(A). 


3. If B is obtained from A by multiplying a row (or column) by a non-zero number 
r and adding the result to another row, then det(B) = det(A). 


Proof. The proof of each of these facts comes from the cofactor expansion of the determinant. 


1. Assume that B is obtained by interchanging the first and second row of A. We will use 
cofactor expansion along the first row to find the determinant of A, and the second row 
for the determinant of B. We get that 

det(A) = S9(-1)any det (Ay) 


j=l 


and 
n 


det(B) = S>(-1)?%bp; det (Boj). 


ji 
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However, since the second row of B is the first row of A, we know that bg; = a1; for 
all 7. In addition, this swap means that we also have that By; = A,; for each of the 
cofactors in this expansion. All of these cofactor matrices are made up of the second 
through last rows of A, with the appropriate columns removed at each step. 


Therefore, the only difference between these two formulas is that the A formula starts 
with (—1)'*7 and the B formula starts with (—1)?*7. Thus, det(B) will have an 
additional factor of —1 in it, giving the desired result. 


The exact same process works for swapping any two adjacent rows of the matrix, giving 
that this also provides a —1 in the computation of the determinant. For non-adjacent 
rows, we use the fact that to any swap of non-adjacent rows of a matrix requires an odd 
number of adjacent row swaps. For example, if we want to swap rows 1 and 3, we can 
swap row 1 with row 2, then row 2 with row 3, and finally swap row 1 with row 2 again. 
This will put the first row in the third spot and the third row up in the first slot. Since 
each of these adjacent switches adds a minus sign, doing an odd number of switches 
still results in adding a single minus sign to the computation of the determinant. 


2. Assume that we want to multiply the &Ath row of A by the number r to get B. We use 
cofactor expansion along this same kth row to find the determinant of each matrix. We 


get that 
det(A) = $0 (-1)*¥ay; det(Ag;) 
j=l 
and o Z 
det(B) = 5 °(-1)*b,; det(Biy) = $0 (-1)* "ray; det (Bay). 
j=l j=l 


However, the minor B;; ignores the kth row of the matrix B, so the minors are identical 
to those of A. Thus, we have that 


n n 


det(B) = S>(-1)* ray; det( Bry) = 7 5 (—1)* 4 agj det (Ags) = 7 det(A). 


j=l j=l 


3. Assume that B is formed by adding r copies of the kth row of A to the ith row. Since 
the ith row is the one being changed, we will use cofactor expansion there to compute 
each determinant. We get that 


n 


det(A) = $0 (-1)'a,; det(Ai; 


j=l 


and 
n 


det(B) = So(-1)"%bi; det(Big) = }0(—1)" (aay + ran) det( Ais) 


j=l 
where we have replaced the minors of B by the minors of A because they ignore the 
ith row, which is the only thing that has changed. We can now split the determinant 
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of B into two parts 


S(-1)7 (aig + rags) det(Aiy) = 5 /(-1)*4aij det(Aiy) + 5 (-1)*? rag; det (Ay). 


The first of these is the determinant of the matrix A. The second is the determinant 
of a new matrix that we will call C. C' is the same as the matrix A, except that we 
have replaced the ith row of A by r times the kth row of A. Thus, the 7th row of this 
matrix C’ is a multiple of the kth row. This means that the rows of C are not linearly 
independent. By Theorem 3.5.4 coming up later (don’t worry, it does not depend on 
this result), this tells us that the determinant of C is zero. Therefore 


det(B) = det(A) + det(C) = det(A) 


so this operation does not change the determinant of the matrix. 


These correspond to the three elementary row operations that we use to row reduce 
matrices. In order to use this to compute determinants, we need to keep track of each of 
these operations and how the determinant changes at each step. 


Example 3.5.1: Compute the determinant of the matrix 


—4 -2 4 
=4.-—3 2 
—2 -3 1 


using row reduction. 


Solution: We will go through the process of row reduction to find the determinant. We need 
to keep track of each time that we swap rows (to add a minus sign) and that we multiply a 
row by a constant (to factor in that constant). Throughout this process, we will use A to 
refer to the initial matrix 


—4 -2 4 
A= |-3 -3 2 
ee | 


and M will refer to wherever we are in the process. So we will start by dividing the first row 
of the matrix by —4 


A ae A t. 12 =f 
=f 3. 0) Sa ae 2 
29. => 7 29 =e Wf 


Since we divided by —4, Theorem 3.5.1 tells us that 


det(M) = -4 det(A). 
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The next step of row reduction will be to use the 1 in the top left to cancel out the —3 and —2 
below it. Part (c) in Theorem 3.5.1 says that this doesn’t change the determinant. Therefore, 
the row reduction gives 


i 17 i f 2. 1 
<8 <8. 2S 10 87> 1 
=9 =3 1 —— 


and we still have that i 
det(M) = =F det (A). 


Next, we will multiply row 2 by —2, which gives 


c 1 1 i 1d: <1 
0-3/2 =1)/)4)0 1° 273 
fb <2) <1 Oa ei 


Adding this in to our previous steps using Theorem 3.5.1, we get that 


jen = (-5) (-7) det(A). 


Finally, we add two copies of row 2 to row 3, which does not change the determinant and 
gives the matrix 


{12 <1 1 12° <1 
0 1 2/3} +]o0 1. 2/3 
0 =o =1 0 0 1/3 


with 


dina (-3) (-7) det(A). 


We can rearrange this expression to say that 


det(A) = 6 det(M) 


and we can easily compute that det(M) = ; by multiplying the diagonal entries. Thus, we 


have that det(A) = 2. | 


Exercise 3.5.1: Compute det(A) using cofactor expansion and show that you get the same 
answer. 


The determinant is telling you how geometric objects scale. If B doubles the sizes of 
geometric objects and A triples them, then AB (which applies B to an object and then it 
applies A) should make size go up by a factor of 6. This is true in general: 


Theorem 3.5.2 


det(AB) = det(A) det(B). 
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This property is one of the most useful, and it is employed often to actually compute 
determinants. A particularly interesting consequence is to note what it means for existence 
of inverses. Take A and B to be inverses, that is AB = I. Then 


det(A) det(B) = det(AB) = det(J) = 1. 


Neither det(A) nor det(B) can be zero. This fact is an extremely useful property of the 
determinant, and one which is used often in this book: 


Theorem 3.5.3 


An n X n matrix A is invertible if and only if det(A) 4 0. 


In fact, det(A~') det(A) = 1 says that 


1 


Se erry 


So we know what the determinant of A~! is without computing A7!. 
Let us return to the formula for the inverse of a 2 x 2 matrix: 


ab) 1 Jd —b 
c d| ad—bc|—-c al}- 


Notice the determinant of the matrix [¢ 5] in the denominator of the fraction. The formula 


only works if the determinant is nonzero, otherwise we are dividing by zero. 


A common notation for the determinant is a pair of vertical lines: 


a b a b 
‘alee ([P a) 
Personally, I find this notation confusing as vertical lines usually mean a positive quantity, 


while determinants can be negative. Also think about how to write the absolute value of a 
determinant. This notation is not used in this book. 


With this discussion of determinants complete, we can now state a major theorem from 
linear algebra that will help us here and when we get back to solving differential equations 
using this linear algebra. In a full course on linear algebra, this theorem would be covered in 
full detail, including all of the proofs. For this introduction, we give some idea as to why 
everything is true here, but not all of the details. 

Note: This is an example of an equivalence theorem, which is fairly common in mathematics. 
It means that if any one of the statements are true, then we know that all of the others 
are true as well. It means it’s harder to prove, but once we have such a theorem, it is very 
powerful in how we can use it going forward. 
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Theorem 3.5.4 
Let A be an n x n matrix. The following are equivalent: 
(a) A is invertible. 


(b) det(A) 4 0. 


) 
) 
(c) There is a unique solution to Av = b for every vector 6. 
(d) The only solution to A? = 0 is ¢ = 0. 
(e) The reduced row echelon form of A is [,,, the identity matrix. 
(f) The rank of A is n. 
) 
) 


g) The rows of A are linearly independent. 


( 
( 


h) The columns of A are linearly independent. 


Proof. Why is all of this true? For (a) and (b), we have Theorem 3.5.3 to say that they are 
equivalent. For (c), if A is invertible, then the unique solution to Av = b is Z = A~‘b. If we 
take b = 0 here, we get (d), that the solution is # = A~10 = 0. This means that reducing the 
system of equations Az = 0 gives x, = 0, ro = 0, ..., ZY, = 0, which means the reduced row 
echelon form of A is just the identity matrix, which is (e). This has n pivot rows, so that the 
rank of A is n. Finally, this means that the dimension of the column space and row space is 
both n, and since there are n of these vectors, it means they are all linearly independent. 


This is a massive theorem that forms most of the backbone of linear algebra. We will 
only be using a few parts of it later, but since we have seen all of the components, it is nice 
to see them all put together into one complete statement. 


3.5.1 Exercises 


Exercise 3.5.2: Compute the determinant of the following matrices: 


123 
1 3 2 1 
a) [3] b) E i c) Fi A d) |0 4 5 
00 6 
as ae 002 -3f fr 1-12 
e) |-2 7 -3 f) |8 6 3 Ola. 4 Bly 4. 5 4 
ne ces Oo 2 4 2 -1 -2 3 


Exercise 3.5.3:* Compute the determinant of the following matrices: 
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2 9 —-11 
2 =2 Dae?) 

a) [—-2| b) F al c) E ‘| d) |O -1 5 

0.0 3 

2 10 513 0020 12-3 4 

e) |-2 7 3 a ee Oa {lo 45 0 bh) |, 6 2 2 

oS er a ae 12 3 -2 


Exercise 3.5.4: For which x are the following matrices singular (not invertible). 


oe Oe 

2 3 oe ae eS al 
of yf 3 ee ali 42 
2 oe L2 4 2 162 
Exercise 3.5.5:* For which x are the following matrices singular (not invertible). 
rae ee 

1 3 3.2 xz 3 
oft oe eee ad |i 40 
lx 1 3 3.2 162 


Exercise 3.5.6:* Consider the matrix 


=i O 
Bees | 5d. 5 
a ee 


a) Compute the determinant of A using cofactor expansion along row 1. 
b) Compute the determinant of A using cofactor expansion along column 2. 
c) Compute the determinant using row reduction. 

Exercise 3.5.7:* Consider the matrix 


he 8 


0 
2. A 
3.03 


A=]1 
3 
a) Compute the determinant of A using cofactor expansion along row 1. 

b) Compute the determinant of A using cofactor expansion along column 3. 
c) Compute the determinant using row reduction. 


Exercise 3.5.8:* Consider the matrix 


2° “Way 

Os Sead) atl 2 
on —5 3 1 383 

—3 4 1 3 
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a) Compute the determinant of A using cofactor expansion along row 1. 
b) Compute the determinant of A using cofactor expansion along column 4. 
c) Compute the determinant using row reduction. 


Exercise 3.5.9: Is the matrix A below invertible? How do you know? 


AO 3. 
ie tae ae 
A=|o 0 1 -3 
2 ae) ae Ass 


Exercise 3.5.10:* Compute the rank of the matrix A below. 


0 -3 2 4 

—5 -4 -5 -l 
a= 1 4 -3 -5 

=e to 2s 


What does this tell you about the invertibility of A? How about the solutions to A¥ = 0? 


Exercise 3.5.11:* Compute the rank of the matrix A below. 


3 —-5 5 
A= |2 -3 3 
4 0 -l 


1 
What does this tell you about the invertibility of A? How about the solutions to Ax = |1| ? 
1 


Exercise 3.5.12:* Compute the determinant of the matrix 


5 4 8 
A= |-4 -3 -4 
—5 -5 4 


using row reduction. What does this say about the solutions to Ax = 0? 


Exercise 3.5.13:* Compute the determinant of the matrix 


=§. =3. —b 1 
4 0 -5 4 
ca Ob -=2: Sh 2 
—-1 -5 -4 -4 


using row reduction. What does this say about the columns of A? 
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Exercise 3.5.14:* Compute the determinant of the matrix 


4 1 -3 0O 
=1° A 2 = 2 
a =1 =e “8 22 
—5 -4 1 1 


1 
using row reduction. What does this say about the solutions to Ax = ] ; 


Exercise 3.5.15: Compute 
= 


21 2 83 
0 8 6 5 
det} 19 0 3 9 
00-0) Qe 
without computing the inverse. 
Exercise 3.5.16:* Compute 
ok Ae I 
0 -1 9 -8 
SHV Gs i Grd, A 
OO.) Oe ~ 2 
without computing the inverse. 
Exercise 3.5.17: Suppose 
10 0 0 5 9 1 —sin(1) 
2 <b 0) 30 0 1 88 —1 
nl ae ae ee Bn Ns: A 3 
22 5 -99 1 0 0 0 1 


Let A = LU. Compute det(A) in a simple way, without computing what is A. Hint: First 
read off det(L) and det(U). 


Exercise 3.5.18: Consider the linear mapping from R* to R? given by the matrix A = [}*] 
for some number x. You wish to make A such that it doubles the area of every geometric 
figure. What are the possibilities for x (there are two answers). 


Exercise 3.5.19 (challenging):* Find all the x that make the matrix inverse 


1 Di: 
L* 
have only integer entries (no fractions). Note that there are two answers. 
Exercise 3.5.20: Suppose A and S aren x n matrices, and S is invertible. Suppose that 


det(A) = 3. Compute det(S~!AS) and det(SAS~'). Justify your answer using the theorems 
in this section. 


Exercise 3.5.21: Let A be ann x n matrix such that det(A) = 1. Compute det(xA) given 
a number «x. Hint: First try computing det(xJ), then note that cA = (ax1)A. 
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3.6 Eigenvalues and Eigenvectors 


After this section, you will be able to: 


e Find the eigenvalues and eigenvectors of a matrix, 

e Use complex numbers to find eigenvalues and eigenvectors if necessary, and 

e Identify the algebraic and geometric multiplicity of an eigenvalue to determine if 
it is defective. 


Consider the matrix 
A 7 -8 
~~ Tee = 


We can compute a few operations with this matrix. For instance 
1 7 —8] {1 —l 
4ll-b =) b]-[o] 
2 7 —8| |2 6 
4bl-6 3] b]-b] 


This last computation is fairly interesting, because the result we get is the same as 3 times 
the original vector. However, the matrix A does not multiply every vector by 3, as seen in 
the first example and the fact that 


and 


4 

Hl , and 
numbers, 3 and 1, are somehow special for this matrix A. With this information, we want to 
define these vectors as eigenvectors and numbers as eigenvalues of the matrix A. 


Definition 3.6.1 


: a ee 2 
so A actually preserves this vector, multiplying it by 1. So, these vectors, H and 


For a square matrix A, we say that non-zero vector U is an eigenvector of the matrix A 
if there exists a number X so that 


In this case, we say that is an eigenvalue of A and it is the corresponding eigenvalue 
for the eigenvector Uv. 
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1 
eigenvector with corresponding eigenvalue 1. 
Why are these important? It turns out that these eigenvalues and eigenvectors characterize 
the behavior of the matrix A. For example, if we wanted to figure out what happens when A 


21 . ; ; : A| , 
we see that | is an eigenvector with corresponding eigenvalue 3, and that A is an 


; 6 
is applied to the vector | , we can figure this out as 


6 4 2 
ald-4(G]+G)) 
4 2 
4b +4b 

4 2 10 

[lsh] [6 
In addition, eigenvectors determine directions in which multiplying by the matrix A 
behaves just like scalar multiplication. This idea will be very important for our understanding 


of systems of differential equations, because we have already seen how to solve a scalar first 
order equation way back in § 0.1 and § 1.3. 


3.6.1 Finding Eigenvalues and Eigenvectors 


Since eigenvalues and eigenvectors are so important, we want to know how to find them. To 
do this, we are looking for a number 4 and a non-zero vector VU so that 


AU = NU. 
We can rewrite this as 
Av — \vU = 0 
or, using the identity matrix, 
(A —Al)v = 0. 


This means that we are looking for a non-zero solution to a homogeneous vector equation of 
the form Bv = 0. This is where all of our linear algebra theory comes into play. 

Theorem 3.5.4 tells us that, combining parts (b) and (d), that there is a non-zero solution 
to (A — Al)v = 0 if and only if the determinant of the matrix A — XI is zero. Therefore, we 
can compute this determinant, find the values of A so that det(A — AJ) = 0, and these will 
give us our eigenvalues. Let’s see an example of what this looks like. 


Example 3.6.1: Compute det(A — AJ) for the matrix 
7 -8 
A= i | 
Solution: For this matrix, we have that 


7 -8 1 0)_[7-A -8 
A-a=|f “31 - [o i =| 3 ao: 
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Thus 


det(a ~ 1) = aet ( [757 Pu) 


= (7 — A)(-3 — A) — (-8)(3) = A? +3A— 7A — 214 24 
= N-4\4+38 
If we were looking for eigenvalues here, we could then set this equal to zero, getting that 
0=-4\4+3= (A-1)(A—3) 
so that the eigenvalues are | and 3. _| 


In this case, we saw that computing det(A — AJ) for this case, we ended up with a 
quadratic polynomial, so it was easy to find the eigenvalues. Thankfully, no matter the size 
of the matrix, we will always get a polynomial here. 


Definition 3.6.2 


For a matrix A, the expression det(A — AJ) is called the characteristic polynomial of 


the matrix. It will always be a polynomial, and for A an n x n matrix, it will be a 
degree n polynomial. 


This explains why we got a quadratic polynomial for the 2 x 2 matrix A. Therefore, for a 
matrix A, the roots of the characteristic polynomial are the eigenvalues of A. 

Once we have the eigenvalues, we can use them to find the eigenvectors. As with how we 
started this discussion, we are looking for a non-zero vector VU so that 


(A—AI)v =0, 


and we know the value of A. Therefore, we can set up a system of equations that correponds 
to 
(A —Alv=0 


and solve it for the components of the eigenvector. 


Example 3.6.2: Find the eigenvalues and eigenvectors of the matrix 
7 -8 
A= : 
5 3 
Solution: The previous example shows that the eigenvalues for this matrix are 1 and 3. For 
the eigenvalue 1, we want to find a non-zero solution to (A — J)v = 0, which means we want 


to solve for 
~.. | =o —8 8, 8) 
(4-Di=| 3 s21[?=|[5 4] #=0. 


Writing the vector v as iW , this system of equations becomes 
2 


6v1 = 85 =0 
3U1 = Avo = 0 
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Since the second equation is two times the first one, these equations are redundant, so we 
only need to satisfy 3, — 4v2 = 0. We can do this by choosing vj = 4 and vg = 3, which 


gives that for \ = 1, a corresponding eigenvector is 3|: 
We can follow the same process for the eigenvalue 3. For this, we want to find a non-zero 
solution to (A — 3/)v¥ = 0, which means that we want to solve 


(a-ana= ("5° me je=[5 5| #=0. 


Writing the vector Vv as , we get the two equations 
2 


Avy = 8v9 =0 
3Uy = 6v5 = 0 


As before, these two equations are the same, since they are both a multiple of vy — 2v2 = 0. 
Therefore, we just need to find a solution to that previous equation, which can be done with 


Vv, = 2 and vo = 1. Therefore, an eigenvector for eigenvalue 3 is Al 


4 


This example illustrates the standard process that is always used to find eigenvalues and 
eigenvectors of matrices: find the characteristic polynomial, get the roots of this polynomial, 
and use each of these eigenvalues to set up a system of equations for the components of each 
eigenvector. In addition, the equations that we get from this system will always be redundant 
if we have found the eigenvalue correctly. Since det(A — AJ) = 0, we know that the rows of 
the matrix A — XJ are not linearly independent, and so the row-echelon form of A — AJ must 
have a zero row in it. This process works for any size matrix, but it becomes harder to find 
the roots of this polynomial when it is higher degree. 


Example 3.6.3: Find the eigenvalues and eigenvectors of the matrix 


ti VQ 
A= 1|9 -4 10 
2 -6 3 


Solution: We start by hunting for eigenvalues by taking the determinant of A — AI, which 
will require the cofactor expansion in order to solve. 
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2. 6 ge) 
-4- 10 9 10 
es oi) as) 


(A? + 4\ — 3A — 12 + 60) — 6(27 — 9A — 20) 
= (1—A)(\7 +24 48) — 424 54d 

=)? +\+48—)? = 0X? —48A — 424 54) 

= —'+7\+6 


1-—A 6 0 
det(A — AI) = det | 9 -4—- xr 10 

) 

) 

) 

) 


We need to look for the roots of this polynomial. There’s no nice way to factor this right 
away, so we need to start guessing roots. We know that the root must be a factor of 6. If we 
try A = 1, we get 


—-1+74+6=120 
so that one doesn’t work. Plugging in A = —1, we get 
—(-1)?} -7+6=1-7+6=0 


so this is a root, meaing that A+ 1 is a factor of the characteristic polynomial. We can then 
use polynomial long division to get that 


—N+7\ +6 = (A+ 1)(-17+A4 6) = -—(A+ D(H - A-6) 


and the quadratic term here factors as (A — 3)(A + 2). Thus, the characteristic polynomial of 
this matrix is 


(A+ 1)(A- 3) 4+ 2) 


so the eigenvalues are —1, 3, and —2. 
For the eigenvalue —1, the eigenvector must satisfy 


(A+D0v=0 


which we can write as 
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To solve this, we row-reduce the coefficient matrix. 


2 6° 3. 0 
9 —3 10} > —3 10 
2-6 4 —6 4 


—12 4 
3 0 
te! 
0 0 


S.S HS  OO SMa  O RR O 
| 
re 
bo 
a 


Therefore, the eigenvector must satsify vj + 3vg = 0 and —3v_2 + v3 = 0. We need to pick 
any non-zero set of numbers that solves these equations. For example, we could pick v2 = 1 
to get that we need v,; = —3 and v3 = 3. This gives an eigenvector of 


—3 
dt 
3 


For the eigenvalue 3, the eigenvector must satisfy 


—2 6 0 Uy 
9 —7 10] |v} =0. 
2 —6 0 U3 


Row reduction gives 


—2 6 O 1 -3 0 
9 -7 10] ~ |9 -—7 10 
2 -6 0 2 -6 0 
1 -3 0 
+  |0 20 10 
Os. 50 30) 
1 -3 0 
> 10 2 1 
0. O° 30 
which means that the eigenvector must satisfy vj — 3v2 = 0 and 2v2+v3 = 0. Again, choosing 
v2 = 1 gives that we want v; = 3 and v3 = —2. Therefore, a corresponding eigenvector here is 
3 
1 


—2 
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For the eigenvalue —2, the eigenvector must satisfy 


3 6 0 Vy 
OF a2* LOH 5 Se) 
2 -—6 5 U3 


where we can row reduce the coefficient matrix. 


3 6 O i gt a 
9 —-2 10; —> |9 -2 10 
2 -6 5 2 -6 5 
1 2 0 
+ |0 —20 10 
0 -—10 5 
1 2 O 
SOs 2. 
0 -10 5 
1 2 O 
= Oe 2 
0 0 O 
Therefore, the eigenvector must satisfy vj + 2v9 = 0 and —2v2 + v3 = 0. Picking v2 = 1 
again gives that we want v; = —2 and v3 = 2. Therefore, an eigenvector with eigenvalue —2 
is 
—2 
1 
2 


=| 


3.6.2 Real Eigenvalues 


Since eigenvalues come from finding the roots of a polynomial, there are a few different 
situations that can arise in terms of these eigenvalues. If we take a quadratic polynomial, 
there are three options for the two roots. 


e Two real and different roots, 
e Two complex roots in a conjugate pair, or 
e One double (repeated) root. 


The same is true for eigenvalues, they are either all real and distinct, there are some that 
appear in complex conjugate pairs, or there are some repeated eigenvalues. The easiest of 
these cases is when the characteristic polynomial has all real and distinct eigenvalues. 

In this case, we get a very nice result. We know that for each eigenvalue, there will always 
be at least one eigenvector, otherwise it wouldn’t be an eigenvalue. If the matrix A is an 
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n Xn matrix, then the characteristic polynomial is a degree n polynomial, which will have 
n distinct roots by our assumption. Each of these will have a corresponding eigenvector, 
giving us n eigenvectors as well. A more involved result tells us that eigenvectors for different 
eigenvalues are always linearly independent. Therefore, we get n vectors in R”, that are 
linearly independent, and so they are a basis. This gives the following result. 


Theorem 3.6.1 


Let A be an n x n matrix. Assume that the characteristic polynomial of A has all real 
and distinct roots, namely that 


= NP) = C= ee = 


for Az, ...,An, the distinct real eigenvalues. Then there exist vectors 0}, ..., Up, such that 
v; is an eigenvector for eigenvalue A; and {vj,...,0,} form a basis of R”. 


To reference, look at the previous example. We found three distinct real eigenvalues of 
—1, 3, and —2. For these eigenvalues, we had eigenvectors 


—3 3 —2 
-l7>} 1 35] 1 -2>)1 
3 —2 2 


These three vectors are linearly independent (check this!) and since they are three component 
vectors, the space has dimension 3, and so 3 linearly independent vectors must make up a 
basis. This is useful to know for now, but will be critical when we want to use this information 
to solve systems of differential equations later. 


3.6.3 Complex Eigenvalues 


When the matrix has complex eigenvalues, the process is very similar to before. However, the 
eigenvector will necessarily also be complex, that is, some of the components of this vector 
will be complex numbers. Let’s illustrate this with an example. 


Example 3.6.4: Find the eigenvalues and eigenvectors of the matrix 
3 —8 
A= ; 7 dl | 


Solution: We first look for the eigenvalues using the characteristic polynomial of A. 


det(a ~ 1) = aet (|? 5 | 


=(3 = A)(—9 =A) +40 
= ji +0\= 3A =—97 +40 
= )\? +6413 
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This quadratic does not factor, so we use the quadratic formula to find that 


\ —6+/67-4-13 -6+~/-16 3-49; 
— — — == i} 
2 2 


so that we have complex eigenvalues. 
We now look for the eigenvectors in the same way as in the real case. If we take the 
eigenvalue —3 + 27, then such an eigenvector must satisfy 


This means that 


(3 297) ~8 g_ [6% =8 Jong 
5 SEs O |S |). Bo Soa oe 


These two equations should be redundant, and to verify that, we will multiply the top 
row by 6 + 27 in row reduction to get 


° aOp 8 | . — 24)(6 + 21) —8(6 + “ul 


5 —6— 2 5 =F = 2) 
40 —48 — 16% 
5 —6—22 


and from this, we can see that the top row is 8 times the bottom one, so they are redundant. 
Thus, an eigenvector must satisfy 


YOR — (6 + 21) v9 = 0 


and we can pick any non-zero numbers that satisfy this. One simple way to do this is by 
switching the coefficients, so that v, = 6 + 22 and vg = 5. Therefore, an eigenvector that we 


get is 
6 + 21 
5 : 


Now, we can take the other eigenvalue, —3 — 27. The process is the same, so that the 
vector must satisfy 


3 =(=3 = 27) —8 ga l6t2% 8) lacy 
5 a9 (3-27) |. Bb Sea oe 


To check redundancy again, we multiply the top row by 6 — 22 to get 
6 + 22 —8 (6 + 27)(6 — 22) —8(6 — 27) 
5 —6 + 2% 5 —6 +4 21 


40 —48 + 162 
9 —6+22 
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and again, the first equation is 8 times the second one. Thus, the eigenvector will need to 


satisfy 
DV1 = (6 = 21) v2 = 0 


which can be done by picking v, = 6 — 27 and v2 = 5, giving an eigenvector of 
6 — 2% 
5° 
= 


The process here is the same as it was in the real case, except that now all of the equations 
are complex equations. In particular, the “redundancy” that we expect to see between the 
equations will likely be via a complex multiple. The easiest way to verify that these equations 
are redundant is by multiplying the first entry in each row by its complex conjugate. This is 
because, if we have the complex number a + bi, multiplying this by a — bi gives 


(a+ bi)(a— bi) = a? + abi-abi- BP? = a? + 0° 


which is now a real number. This will make it easier to compare the two equations to make 
sure that they are redundant, and that the eigenvalue was found correctly. 


Example 3.6.5: Find the eigenvalues and eigenvectors of the matrix 


LBS 56 
A= 0 1: 26 
0 =3- 5 


Solution: We first look for eigenvalues, like always. We get these by computing 


1-2 9 6 
det(A — AI) = det 0 1-A 6 
0 —-3 -5-X 


We will compute this by cofactor expansion along the second row. 


1-2 9 6 
elie 1-6 
det 0 1-A 6 = (-1)°"(1 a) act (| 6 Lee 


+ (—1)?6 det (is i 
(SA) A) So-= A) = 00 = A)(=3) 


= (1 — A)((1 — A)(—5 — A) + 18) 
= (1—A)(A? + 4A 4+ 18) 


so that one eigenvalue is at \ = 1. For the other two, we use the quadratic formula to obtain 


_-44/16-4-13_ -4+ /—36 


2 2 


A S223, 
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Thus, we have one real eigenvalue and two complex eigenvalues. 
For \ = 1, we know that the eigenvector must satisfy 


0 9 6 
0 0 6]%0=0. 
0 -—3 —6 
Row reduction will reduce this matrix to 
0 1 0 
00 1 
00 0 


(Check this!) so that the eigenvector in this case is 


For the eigenvalue —2 + 32, we get that the eigenvector must satisfy 


We now want to row reduce the coefficient matrix. To do so, we start by dividing the first 
row by 3 then multiplying by 1 +7. 


2230 <6 6 Gee: 3 2 
i Bete . 6. pee Or eee 6 
0 <3. 2353) Oc (8 S33) 

= G7) 802). Bs) 

= 0 B23) BH 

0 23. 333) 


2k edt. D224 
> 10 3-3 6 
0 -3 —3—32 


We could divide the first row by 2 to get to a 1 in the top-right entry, but we’ll wait on that 
in order to avoid fractions. To row reduce the rest of the matrix, we will divide each of the 
remaining two rows by 3, and then multiply the second by 1 + 7, just like we did to the first 
row. 
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2 d3+dt 2423 2 3$+32 2422 
0 3-31 6 Sri dle 2 2 

0 -3 —3—-—31 OG. =. bz 

2 3+32 2422 

— 10 2 25521 

O. =ff ‘=]=a 

2 $+3¢ 2422 

—+ {0 1 eS 

OQ. cei: =lag 


which illlustrates that the last two rows are redundant. Thus, the reduced form of the matrix 
that we have (which is not quite a row echelon form, but it is enough to back-solve for the 
eigenvector) is 

2 3431 2421 

0 ih 1+2 

0 0 0 


This means that the eigenvector Vv must satisfy 


2u, + (3 + 32)ve + (2 + 27)v3 = 0 Vo + (1+ 12)v3 = 0. 


We can satisfy the second of these equations by choosing v2 = 1 +7 and v3 = —1. Plugging 
these values into the first equation gives that 
0 = 2v, + (3 + 82)u2 + (2 + 22) u3 
= 2v, + (384+ 32)(1 +2) + (2 + 27)(-1) 
= 20, +34+314+ 31-3 -—2-— 21 
= 2v, —2+ 41 


Therefore, we need to take v; = 1 — 27, giving that the eigenvector is 


1-23 
Lot 
—1 


A very similar computation following the same set of steps (or just using the remark 
below) for the eigenvalue —2 — 37 gives that this corresponding eigenvector is 


1+ 21 
1-i 
—1 
= 
One fact that comes out of those examples is that the eigenvectors for conjugate eigenvalues 
are also complex conjugates. This comes from the fact that A is a real matrix, which means 


that if 
Atv = Uv 
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and we take the complex conjugate of both sides, we get that 


At = AU = UW = NU 


so that @ is an eigenvector for \. This means that, when solving these types of problems, 
we only need to find one of the complex eigenvectors and can get the other by taking the 
complex conjugate. 


3.6.4 Repeated Eigenvalues 


Distinct and complex eigenvalues all work out nicely and in pretty much the same manner. 
For repeated eigenvalues, the issues get more significant. 


Example 3.6.6: Find the eigenvalues and eigenvectors of the matrices 
a 20) 4 —1 
a= s] 8-[ a]: 
Solution: For the matrix A, we can compute the characteristic polynomial 


det(A — AI) =aet (|? 9” al) =(3-A)8=.) 


Therefore, we have a double root at 3 for this matrix. Therefore, the only eigenvalue we 
get is 3. When we look to find the eigenvectors, we get 


0 0 
a-ar=fP 9 


so that this matrix multiplied by any vector is zero. Therefore, we can use both i and H 


as eigenvectors. 
On the other hand, the matrix B has a characteristic polynomial 


det(B — AI) = det a “4 : el) 


= (4—A)(2—A) — (-1)0) = -6A4+8 41 
= -64+9= (A- 3)? 


so again, we have a double root at 3. However, when we go to find the eigenvectors, we get 


that 
1 -l 
B-3I = | 


1 works. 


which gives that an eigenvector must satisfy vj — vo = 0 so i 


_| 


There is a big difference between these two examples. Both had the same characteristic 
polynomial of (A — 3)?, but for A, we could find two linearly independent eigenvectors, but 
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for B, we could only find 1. This seems like it might be a problem, since we would like to get 
to two eigenvectors like we did for both of the previous two cases. This leads us to define the 
following for A and n x n matrix and r an eigenvalue of A. 


Definition 3.6.3 


The algebraic multiplicity of r is the power of (A — r) in the characteristic 
polynomial if A. 


The geometric multiplicity of r is the number of linearly independent eigenvectors 


of A with eigenvalue r. 


The defect of r is the difference between the algebraic multiplicity and the 
geometric multiplicity of r. 


We say that an eigenvalue is defective if the defect is at least 1. 


For the previous example, the algebraic multiplicity of 3 for both A and B was 2, but the 
geometric multiplictiy of 3 for A is 2, and for B is it only 1. Therefore A has a defect of 0 
and B has a defect of 1, so 3 is a defective eigenvalue for matrix B. 

In terms of these multiplicities, there are two facts that are known to be true. 


1. If r is an eigenvalue, then both the algebraic and geometric multiplicity are at least 1. 


2. The algebraic multiplicity of any eigenvalue is always greater than or equal to the 
geometric multiplicity. 


This tells us that in the case of real and distinct eigenvalues, every eigenvalue has 
multiplicity 1. Since the geometric multiplicity is also 1, this means that none of these 
eigenvalues are defective. This was great, because it let us get to n eigenvectors for an n x n 
matrix, and these generated a basis of R”. 

Why is a defective eigenvalue a problem? When we go solve differential equations using 
the method in Chapter 4, having a ‘full set’ of eigenvectors, or n eigenvectors for an n x n 
matrix, will be very important. When we have a defective eigenvalue, we can’t get there. 
Since the degree of the characteristic polynomial is n, the only way we get to n eigenvectors 
is if every eigenvalue has a number of linearly independent eigenvectors equal to the algebraic 
multiplicity, which means they are not defective. 

So how can we fix this? Well, there’s not really much we can do in the way of finding more 
eigenvectors, because they don’t exist. The replacement that we have is, in linear algebra 
contexts, called a generalized eigenvector. We will see this idea come back up in § 4.6 in a 
more natural way. The rest of this section contains a more detailed definition of generalized 
eigenvectors. You are welcome to skip this part on a first reading and come back after you 
are more comfortable with eigenvalues and eigenvectors, or when the material comes back 
around again in § 4.6. 

If r is an defective eigenvalue of the matrix A with eigenvector v, a generalized eigenvector 
of A is a vector w so that (A—rJ)w =v. This is the same as the normal eigenvector equation 
with @ on the right-hand side instead of 0. Since (A — rl) = 0, this also means that 


(A—rI)a =0. 
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More generally, a generalized eigenvector is a vector Ww where there is a power k > 1 so that 
(A—rI)*a =0 but (A—rI)* 1H £0. 
It might seems strange where this comes from, but we will see why this formula makes 
more sense once we try to solve differential equations using matrices in § 4.6. 


Example 3.6.7: Find a generalized eigenvector of eigenvalue 3 for the matrix 
4 —1 
B= i ; | | 
1 : : 
1 is an eigenvector for B with eigenvalue 3. To 
find a generalized eigenvector, we need a vector w so that 


Solution: Previously, we found that 0 = 


G=snge H 


Plugging in the matrix for B — 3/ gives that we need 


{. =1) fon] [1 
1 —1} |we| {1} 
Both of the rows of this matrix becomes the equation 


W1— We = 1. 


There are many values of w; and w2 that make this work. We can pick w,; = 1 and wz = 0. 
1 


i): We could also pick w; = 3 and w» = 2, to get 


This will give a generalized eigenvector of 


a generalized eigenvector as A . Any of these choices work as a generalized eigenvector. 


2 


Example 3.6.8: Find the eigenvalues and eigenvectors (and generalized eigenvectors if 
needed) of the matrix 


—2 0 1 
A=|19 2 -16 
-1 0 0O 
Solution: We start by looking for the eigenvalues through the characteristic polynomial. 
—2-A 0 1 
det(A — AI) = det 19 2-—A —16 


=I 0 O-—A 


To compute this determinant, we will expand along column 2, because it only has one non-zero 
entry. This gives 


det(A — AI) = (—1)?*7(2 — A) det (5 : ‘) 


= (2—A)((-2— A)(-A) + 1) 
= (2—A)(M*7+2A41) = (2-A)A+1)? 
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so we have an eigenvalue at 2 and a double eigenvalue at —1. 
First, let’s look for the eigenvector for eigenvalue 2. In this case, we know that the 
eigenvector must satisfy 


SAG 
19 0 —16] *¢=0. 
=e Oe 29 


Row reducing the coefficient matrix will give 


b 0 
00 1 
0°00 
so that a corresponding eigenvector is 
0 
1 
0 
since we know that v; = 0 and v3 = 0. 
For \ = —1, we see that an eigenvector must satisfy 
-1 0 1 
19 3 -16] ¢=0. 
-1 0 1 


We now look to row reduce this coefficient matrix. 


-1 0 1 1 O -1 
19 3 -16] > |19 3 —16 
-1 0 1 -1 0 1 
1 0 -l 
+> |0 3 3 
O20). 20 
1 0 -l 
Spr Oa, ah 
00 O 
Therefore, we know that 
v1 — v3 = 0 Vg + v3 = 0. 
If we pick v3 = 1, then we know that vo = —1 and v; = 1, so the only eigenvector we get for 
A= —-lis 
1 
—1 
1 
Since we only found one eigenvector for \ = —1 and A+1 was squared in the characteristic 


polynomial, this is a defective eigenvalue. Thus, we can look for a generalized eigenvalue 
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here, which means that we need to solve for a vector w with 


=—1 0 J 1 
19 3 —16] #@= |-1 
=1 20. ot 1 


We can then row reduce the augmented matrix to see what we can pick for w. 


=H 1 Le oie st - S 
TOs do 16: AHL) Set 1 3 S16 
=—1 0 1 1 et 1 

MO Sah al 

> {103 3 18 

OO. 1. oO 

Ly) eh 1 

> )]0 1 1 6 

0: <0 Or <Q 


Thus, the generalized eigenvector w must satisfy 
W, —w3 = -1 Wo + Ws = 6. 


We can pick any non-zero numbers to do this, so we can take w3 = 1, wo = 5 and w, = 0. 
Thus, the generalized eigenvector here is 


3.6.5 Exercises 


Exercise 3.6.1:* Find the eigenvalues and corresponding eigenvectors of the matrix 
—8 -18 
4 10 
Exercise 3.6.2:* Find the eigenvalues and corresponding eigenvectors of the matrix 
—2 0 
8 —4 


Exercise 3.6.3:* Find the eigenvalues and corresponding eigenvectors of the matrix 


eal 
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Exercise 3.6.4:* Find the eigenvalues and corresponding eigenvectors of the matrix 
—3 5 
—8 9 
Exercise 3.6.5:* Find the eigenvalues and corresponding eigenvectors of the matrix 
O 2 
—l -2 
Exercise 3.6.6:* Find the eigenvalues and corresponding eigenvectors of the matrix 
—4 1 
—8 0 
Exercise 3.6.7:* Find the eigenvalues and corresponding eigenvectors of the matrix 
0 -8 
2>. 98 
Exercise 3.6.8:* Find the eigenvalues and corresponding eigenvectors of the matrix 
1 -—2 
8 —7 
Exercise 3.6.9:* Find the eigenvalues and corresponding eigenvectors of the matrix 
0 0 
-4 2 1 
-6 0 1 


Exercise 3.6.10:* Find the eigenvalues and corresponding eigenvectors of the matrix 


—4 9 9 
—3 6 9 
o- =. S10 


Exercise 3.6.11:* Find the eigenvalues and corresponding eigenvectors of the matrix 


mz 0h, 0) 
0 4 6 
b- SoiS2 


Exercise 3.6.12:* Find the eigenvalues and corresponding eigenvectors of the matrix 
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Exercise 3.6.13:* Find the eigenvalues and eigenvectors for the matrix below. Compute 
generalized eigenvectors if needed to get to a total of two vectors. 


—ll -9 
Te 10 


Exercise 3.6.14:* Find the eigenvalues and eigenvectors for the matrix below. Compute 
generalized eigenvectors if needed to get to a total of two vectors. 


4 —4 

1 O 
Exercise 3.6.15: This exercise will work through the process of finding the eigenvalues and 
corresponding eigenvectors of the matrix 


—2 0 -8: 
Aa TDs oar. 12 
0: lL. a 


a) Find the characteristic polynomial of this matrix by computing det(A — AI). In finding 
this, use cofactor expansion along either column 1 or row 1 and do not expand out all 
of the terms. Use grouping to factor this polynomial. 


b) This polynomial can be rewritten as —(A—r,)?(A—r2) where r, and rz are the eigenvalues 
of A. What are the eigenvalues? What is each of their algebraic multiplicity? 


c) Find an eigenvector for eigenvalue rz above. What is the geometric multiplicity of this 
eigenvalue? 


d) Find an eigenvector for eigenvalue r;._ What is the geometric multiplicity of this 
eigenvalue? 


e) There is only one possible eigenvector for r,, which means it is defective. Find a solution 
to the equation (A — r,I)wW = v, where V is the eigenvector you found in the previous 
part. This is the generalized eigenvector for r,. 


Exercise 3.6.16: We say that a matrix A is diagonalizable if there exist matrices D and 
P so that PDP~! = A. This really means that A can be represented by a diagonal matrix 
in a different basis (as opposed to the standard basis). One way this can be done is with 
eigenvalues. 


a) Consider the matrix A given by 


a= [7 9). 


Find the eigenvalues and corresponding eigenvectors of this matrix. 
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b) Form two matrices, D, a diagonal matrix with the eigenvalues of A on the diagonal, 
and E, a matrix whose columns are the eigenvectors of A in the same order as the 
eigenvalues were put into D. Write out these matrices. 


c) Compute E7". 
d) Work out the products EDE~' and E~!AE. What do you notice here? 


This shows that, in the case of a 2 x 2 matrix, if we have two distinct real eigenvalues, that 
matrix is diagonalizable, using the eigenvectors. 


Exercise 3.6.17: Follow the process outlined in Exercise 3.6.16 to attempt to diagonalize 
the matrix 


13 14 «+12 
—6 -4 -6 
=a 6. 2 


Hint: 1 is an eigenvalue. 


Exercise 3.6.18: The diagonalization process decribed in Exercise 3.6.16 works for any case 
where there are real and distinct eigenvalues, as well as complex eigenvalues (but the algebra 
with the complex numbers gets complicated). It may or may not work in the case of repeated 
eigenvalues, and it fails whenever there are defective eigenvalues. Consider the matrix 


4 -1 
I 2 
a) Find the eigenvalue(s) of this matrix, and see that we have a repeated eigenvalue. 


b) Find the eigenvector for that eigenvalue, as well as a generalized eigenvector. 


c) Build a matrix EF like before, but this time put the eigenvector in the first column and 
the generalized eigenvector in the second. Compute E~'. 


d) Find the product E~'AE. Before, this gave us a diagonal matrix, but what do we get 
now? 


The matrix we get here is almost diagonal, but not quite. It turns out that this is the best 
we can do for matrices with defective eigenvalues. This matrix is often called J and is the 
Jordan Form of the matrix A. 


Exercise 3.6.19:* Follow the process in Exercise 3.6.18 to find the Jordan Form of the 
matrix 
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3.7 Related Topics in Linear Algebra 


Attribution: [JL], §A.4. 


After this section, you will be able to: 


Determine the kernel of a matrix using row reduction, 

Understand the connection between rank and nullity in a given matrix, 
Compute the inverse of a matrix using row reduction, and 

Use properties of the trace and determinant to analyze the eigenvalues of a 
matrix. 


3.7.1 Kernel 


The set of solutions of a linear equation Li’ = 0, the kernel of L, is a subspace: If Z and 7 
are solutions, then 


L(é+y9)=L€+L¥=04+0=0, and  Ll(a#)=aL%=a0=0. 


So #+y and aZ are solutions. The dimension of the kernel is called the nullity of the matrix. 

The same sort of idea governs the solutions of linear differential equations. We try to 
describe the kernel of a linear differential operator, and as it is a subspace, we look for a 
basis of this kernel. Much of this book is dedicated to finding such bases. 

The kernel of a matrix is the same as the kernel of its reduced row echelon form. For a 
matrix in reduced row echelon form, the kernel is rather easy to find. If a vector 7 is applied 
to a matrix L, then each entry in % corresponds to a column of L, the column that the entry 
multiplies. To find the kernel, pick a non-pivot column make a vector that has a —1 in the 
entry corresponding to this non-pivot column and zeros at all the other entries corresponding 
to the other non-pivot columns. Then for all the entries corresponding to pivot columns make 
it precisely the value in the corresponding row of the non-pivot column to make the vector 
be a solution to L# = 0. This procedure is best understood by example. 


Example 3.7.1: Consider 


This matrix is in reduced row echelon form, the pivots are marked. There are two non-pivot 
columns, so the kernel has dimension 2, that is, it is the span of 2 vectors. Let us find the 
first vector. We look at the first non-pivot column, the 2"¢ column, and we put a —1 in the 
2"¢ entry of our vector. We put a 0 in the 5'" entry as the 5 column is also a non-pivot 


262 CHAPTER 3. LINEAR ALGEBRA 


column: 


Let us fill the rest. When this vector hits the first row, we get a —2 and 1 times whatever the 
first question mark is. So make the first question mark 2. For the second and third rows, it 
is sufficient to make it the question marks zero. We are really filling in the non-pivot column 
into the remaining entries. Let us check while marking which numbers went where: 


1 00 3 0 
0 (0) 1 0 4] |(0)| = Jo 
0 015 0 
Yay! How about the second vector. We start with 

? 

0 

: 

- 

-1. 


We set the first question mark to 3, the second to 4, and the third to 5. Let us check, marking 
things as previously, 


—l 


There are two non-pivot columns, so we only need two vectors. We have found the basis of 
the kernel. So, 


kernel of L = span 0 
0 
0 


What we did in finding a basis of the kernel is we expressed all solutions of LZ = 0 as a 
linear combination of some given vectors. 
The procedure to find the basis of the kernel of a matrix L: 


(i) Find the reduced row echelon form of L. 
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(ii) Write down the basis of the kernel as above, one vector for each non-pivot column. 


The rank of a matrix is the dimension of the column space, and that is the span on the 
pivot columns, while the kernel is the span of vectors one for each non-pivot column. So the 
two numbers must add to the number of columns. 


Theorem 3.7.1 (Rank—Nullity) 


If a matrix A has n columns, rank r, and nullity & (dimension of the kernel), then 


n=rt+k. 


The theorem is immensely useful in applications. It allows one to compute the rank r if 
one knows the nullity k and vice versa, without doing any extra work. 

Let us consider an example application, a simple version of the so-called Fredholm 
alternative. A similar result is true for differential equations. Consider 


Ag = b, 
where A is a square n x n matrix. There are then two mutually exclusive possibilities: 
(i) A nonzero solution # to AZ = 0 exists. 
(ii) The equation A# = b has a unique solution Z for every 6. 


How does the Rank—Nullity theorem come into the picture? Well, if A has a nonzero solution 
# to Az = 0, then the nullity & is positive. But then the rank r = n — k must be less than 
n. In particular it means that the column space of A is of dimension less than n, so it is a 
subspace that does not include everything in R”. So R” has to contain some vector b not in 
the column space of A. In fact, most vectors in R” are not in the column space of A. 

The idea of a kernel also comes up when defining and discussing eigenvectors. In order to 
find this vector, we are looking for a vector U so that 


(A-—AD¢=0. 


This means that we are looking for a vector U that is in the kernel of the matrix (A — AJ). 
Since the kernel is also a subspace, this means that the set of all eigenvectors of a matrix A 
with a certain eigenvalue is a subspace, so it has a dimension. This dimension is number of 
linearly independent eigenvectors with that eigenvalue, so it is the geometric multiplicity of 
this eigenvalue. This also motivates why this is sometimes called the ezgenspace for a given 
eigenvalue. Finding a basis of this subspace (which is also finding the kernel of the matrix 
A — XI ) is the exact same as the process of finding the eigenvectors of the matrix A. 


3.7.2 Computing the inverse 


If the matrix A is square and there exists a unique solution # to Av = b for any b (there are 
no free variables), then A is invertible. 
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In particular, if AZ = b then Z = A~'b. Now we just need to compute what A-! is. We 
can surely do elimination every time we want to find A710, but that would be ridiculous. The 
mapping A! is linear and hence given by a matrix, and we have seen that to figure out the 
matrix we just need to find where does A~! take the standard basis vectors €), €), ..., En. 

That is, to find the first column of A~! we solve Ax = €;, because then A7'é, = Z. To 
find the second column of A~! we solve AZ = €2. And so on. It is really just n eliminations 
that we need to do. But it gets even easier. If you think about it, the elimination is the same 
for everything on the left side of the augmented matrix. Doing n eliminations separately we 
would redo most of the computations. Best is to do all at once. 

Therefore, to find the inverse of A, we write an n x 2n augmented matrix | A | I], where 
I is the identity matrix, whose columns are precisely the standard basis vectors. We then 
perform row reduction until we arrive at the reduced row echelon form. If A is invertible, 
then pivots can be found in every column of A, and so the reduced row echelon form of 
[A | I] looks like [J | A~']. We then just read off the inverse A~!. If you do not find a pivot 
in every one of the first n columns of the augmented matrix, then A is not invertible. 

This is best seen by example. 


Example 3.7.2: Find the inverse of the matrix 
Le De 8 


2: 30h aL 
3 1 0 


Solution: We write the augmented matrix and we start reducing: 


Ww 
28 te 2 33. 2050 Bde O50 
201/010] >] 0 -4 -5/-2 10] 3] 0 Q) %/}% 14 0 
OO uGs oe O° S07. 9.3: 0 OG. =o: - =O) je iO Tl 
% “3 Th. .20,:7.0 ane a a Gs 6 a 
5/4 | Yo Ya O| >] O 5/4} V2 Ya 0 > 
QO 1/4} -Y2 -5/4 1 0 0 2/ta 5/11 4/11 
22-40 |e See, eae OQ.) Yar - S/ay. --2/rr 
>] 0 0 )3%u -%1 51 | > |] O 0 | 311 ~%11 5/11 
0 0 2/11 Bf/ta 4/11 0 0 afar fae: - Ala 


1 263 Sid) > Sti 21 
2 Oe =] 3% —%11 5/11 
3 1 0 


Aide, Pf Aft 
a | 4 
Not too terrible, no? Perhaps harder than inverting a 2 x 2 matrix for which we had a 


formula, but not too bad. Really in practice this is done efficiently by a computer. 


3.7.3. Trace and Determinant of Matrices 


The next thing to add into our toolbox of matrices is the idea of the trace of a matrix, and 
how it and the determinant relate to the eigenvalues of said matrix. 
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Definition 3.7.1 


Let A be an n x n square matrix. The trace of A is the sum of all diagonal entries of A. 


For example, if we have the matrix 


the trace is 1+2+3=6. 

The trace is important in our context because it also tells us something about the 
eigenvalues of a matrix. To work this out, let’s consider the generic 2 x 2 matrix and how we 
would find the eigenvalues. If we have a 2 x 2 matrix of the form 


a b 
— 
F 
we can write out the expression det(A — AJ) in order to find the eigenvalues. In this case, we 


would get 


b 
d—x 


eiA—ah Sa |\o-* Sen DGo i tte Gaited Fe. 
(a) 


However, the coefficients in this polynomial look familiar. (ad— bc) is just the determinant 
of the matrix A, and a +d is the trace. Therefore, for any 2 x 2 matrix, we could write the 
characteristic polynomial as 


det(A — AJ) = # —TA+ D (3.3) 


where T is the trace of the matrix and D is the determinant. On the other hand, assume that 
r, and rg are the two eigenvalues of this matrix (whether they be real, complex, or repeated). 
In that case, we know that this polynomial has r; and rz as roots. Therefore, it is equal to 


det(A — AI) = (A—1i)(A—1re) = XV — (rp tre)A t+ rire. (3.4) 


Matching up the coefficient of A and the constant term in (3.3) and (3.4) gives the relation 
that 
T=r,+1%2 D=rjYo, 
that is, the trace of the matrix is the sum of the eigenvalues, and the determinant of the 
matrix is the product of the eigenvalues. We only showed this fact for 2 x 2 matrices, but it 
does hold for matrices of all sizes, giving us the following theorem. 


Theorem 3.7.1 


Let A be an nxn square matrix with eigenvalues 1, Ag, ..., An, written with multiplicity 
if needed. Then 


(a) The trace of A is Ay + Ag +--+ + An. 


(b) The determinant of A is (A1)(A2) +++ (An). 
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From the above statement, we note that if any of the eigenvalues is zero, the product of 
all eigenvalues will be zero, and so the matrix will have zero determinant. This gives an extra 
follow-up fact, and addition to Theorem 3.5.4. 


Theorem 3.7.2 


A matrix A is invertible if and only if all of it’s eigenvalues are non-zero. 


Example 3.7.3: Use the facts above to analyze the eigenvalues of the matrix 
1 2 
ro Ee | | 

Solution: From the matrix A, we can compute that the trace of A is 1 +4 = 5, and the 
determinant is (1)(4) — (2)(5) = —6. Based on the theorem above, we know that the two 
eigenvalues of this matrix must add to 5 and multiply to —6. While you could probably guess 
the numbers here, the important take-aways from this example are what we can learn. 

The main fact to point out is that this is enough information, in the 2 x 2 case, to tell us 
that the eigenvalues have to be real and distinct. Since their product is a negative number, 


we can eliminate the other two options. If we have two complex roots, they must be of the 
form «+ iy and x — iy, and so the product is 


(x + iy)(x — iy) = 2? + izy —izy-—@Py =a? +y° 
which is always positive, no matter what x and y are. Similarly, if we have a repeated 
eigvalue, the product will be that number squared, which is also positive. Therefore, if the 
determinant of a 2 x 2 matrix is negative, the eigenvalues must be real and distinct, with one 


being positive and one negative (otherwise the product can not be negative). These facts will 
be important when we start to analyze the solutions to systems of differential equations in 


Chapter 4. | 
Example 3.7.4: What can be said about the eigenvalues of the matrix 
0 -1 0O 
A= | 2 2 0 |? 
—7 -3 -1 


Solution: We can find the same information as the previous example. The trace of A is 1, 
and the determinant, by cofactor expansion along column 3, is (—1)(0 + 2) = —2. Therefore, 
the sum of the three eigenvalues is 1, and the product of them is —2. We don’t actually have 
enough information here to determine what the eigenvalues are. The issue is that with three 
eigenvalues, there are many different ways to get to a product being negative. There could 
be three negative eigenvalues, two positive and one negative, or one negative real with two 
complex eigenvalues. However, the one thing we do know for sure is that there must be one 
negative real eigenvalue. For this particular example, we can compute that the eigenvalues 
are —1, 1+7, and 1 —2, so we did end up in the complex case. | 


Exercise 3.7.1: Imagine that we have a3 x 3 matrix with a positive determinant (it doesn’t 
matter what the trace is). Think about all the scenarios and verify that at least one eigenvalue 
must be real and positive for this to happen. 
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3.7.4 Exercises 


Exercise 3.7.2: For the following matrices, find a basis for the kernel (nullspace). 


1 tha Oe- SA) 223 444 a9 78 od 
a) {1 1 5 Bde 0> a ea ah Gy) (Shey 2.2 
Lol 4 = ar —5 5 5 1 04,2 


Exercise 3.7.3:* For the following matrices, find a basis for the kernel (nullspace). 


2619 D2 5 i oe ea 044 
Ayu 3° 2e°6 b) {|-1 1° 5 c) |2 3 5 d)|0 11 
3909 a5 Re eS =3:. 5 @ 055 


Exercise 3.7.4: Suppose a5 x 5 matrix A has rank 3. What is the nullity? 


Exercise 3.7.5: Consider a square matrix A, and suppose that X is a nonzero vector such 
that Ax = 0. What does the Fredholm alternative say about invertibility of A? 


Exercise 3.7.6: Consider 


If the nullity of this matrix is 2, fill in the question marks. Hint: What is the rank? 
Exercise 3.7.7:* Suppose the column space of a9 x 5 matrix A of dimension 3. Find 
a) Rank of A. b) Nullity of A. 
c) Dimension of the row space of A. d) Dimension of the nullspace of A. 


e) Size of the maximum subset of linearly 
independent rows of A. 


Exercise 3.7.8: Compute the inverse of the given matrices 


1 0 0 111 1 2 3 

a) |0 0 1 by. 10 oe a OC). 2s 

Os ak 3G a aa On 22: 
Exercise 3.7.9:* Compute the inverse of the given matrices 

0 1 0 111 2 4 0 

a) |-1 0 0 b) |}1 1 0 O12. 2. B 

0 O01 1 0 0 241 


Exercise 3.7.10: By computing the inverse, solve the following systems for <. 


of Jets of deLs 
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Exercise 3.7.11:* By computing the inverse, solve the following systems for Z. 
as ee) 7 aed an 6 
oa al?=(d mt l= 
Exercise 3.7.12:* For each of the following matrices below: 


a) Compute the trace and determinant of the matrix, and 


b) Find the eigenvalues of the matrix and verify that the trace is the sum of the eigenvalues 
and the determinant is the product. 


(i) i | (ii) F a twit) ee eal Gy ee #1 


Exercise 3.7.13:* For each of the following matrices below: 
a) Compute the trace and determinant of the matrix, and 


b) Find the eigenvalues of the matrix and verify that the trace is the sum of the eigenvalues 
and the determinant is the product. 


—1 -16 -4 if 2 0 10 —7 —14 


i) }1 6 1 Ga) 12? Sie =4 (i?) |0 5 6 
= a Ae 16 14 3 7 8.14 


Chapter 4 


Systems of ODEs 


4.1 Introduction to systems of ODEs 


Attribution: [JL], §3.1. 


After this section, you will be able to: 


e Classify the order and number of components in a system of differential equations, 


e Verify if a set of functions solves a system of differential equations, and 
e Write a system of differential equations to fit a physical situation. 


4.1.1 Systems 


Often we do not have just one dependent variable and one equation. For instance, we may 
be looking at multiple populations that are changing over time, or watching how the amount 
of support for multiple candidates develops leading up to an election. And as we will see, we 
may end up with systems of several equations and several dependent variables even if we 
start with a single equation. 

If we have several dependent variables, suppose y1, y2, --., Yn, then we can have a 
differential equation involving all of them and their derivatives with respect to one independent 
variable x. For example, y{ = f(yi,y5, 41; Y2,v). Usually, when we have two dependent 
variables we have two equations such as 


yi = fi(Yis Yo, Yt Y2; £), 
Ys = fo(Y1, Yas Vis Y2, Z), 


for some functions f; and fg. We call the above a system of differential equations. More 
precisely, the above is a second order system of ODEs as second order derivatives appear. 
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The system 


1(@1, £2, £3, t), 
(x1, 2, U3, ty: 


2 
3(@1, £2, £3, t), 


/ 
%=9 
/ 
T= 9 
/ 
T= 9 


is a first order system, where x1, £2, #3 are the dependent variables, and t is the independent 
variable. 

The terminology for systems is essentially the same as for single equations. For the system 
above, a solution is a set of three functions 21(t), xo(t), 73(t), such that 


tlt) = 91 (x1(t), vo(t), x3(t), ¢), 
5(t) = 92 (xi(t), vo(t), 73(t), ¢), 
x3(t) = g3(v1(t), a(t), va(t), t). 


In order to verify that something is a solution, we plug the different components into the 
solution to see that all of the equations are satisfied; if any one of the equations is not satisfied, 
then this set of functions is not a solution. We usually also have an initial condition. Just 
like for single equations we specify x1, 2, and x3 for some fixed t. For example, x7;(0) = a1, 
X2(0) = ag, x3(0) = ag for some constants a1, a2, and a3. For the second order system we 
would also specify the first derivatives at that same initial time point. And if we find a 
solution with constants in it, where by solving for the constants we find a solution for any 
initial condition, we call this solution the general solution. Best to look at a simple example. 


Example 4.1.1: Sometimes a system is easy to solve by solving for one variable and then 
for the second variable. Take the first order system 


V4 = 41; 
Yo = Y1 — Ye, 


with yi, y2 as the dependent variables and x as the independent variable. Consider initial 
conditions y,(0) = 1, y2(0) = 2 and solve the initial value problem. 


Solution: We note that y; = Ce” is the general solution of the first equation, which we can 
get because this equation does not involve yo at all and we can get a solution via our normal 
first order equation methods. We then plug this y; into the second equation and get the 
equation ys = Ce” — ye, which is a linear first order equation that is easily solved for y2. By 
the method of integrating factor we get 


C 
eC" Yo i ae" 7 Co, 
2 
or Yo = “et + Cye~*. The general solution to the system is, therefore, 


6: 
v= Cie’, Y2 = ria + Coe. 


We solve for C and C2 given the initial conditions. We substitute « = 0 and find that C; = 1 
and C = 3/2. Thus the solution is y; = e”, and y2 = (1/2)e” + (3/2)e*. 
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Generally, we will not be so lucky to be able to solve for each variable separately as in the 
example above, and we will have to solve for all variables at once. While we won’t generally 
be able to solve for one variable and then the next, we will try to salvage as much as possible 
from this technique. It will turn out that in a certain sense we will still (try to) solve a bunch 
of single equations and put their solutions together. Let’s not worry right now about how to 
solve systems yet. 

We will mostly consider linear systems. The example above is an example of a linear first 
order system. It is linear as none of the dependent variables or their derivatives appear in 
nonlinear functions or with powers higher than one (2, y, x’ and y’, constants, and functions 
of t can appear, but not xy or (y')? or x*). A more complicated example of a second order 
linear system is 


yf =e'y, + yy + 5y2 + sin(t), 
yg = ty, — yo + 2y1 + cos(t). 


4.1.2 Applications 
Let us consider some simple applications of systems and how to set up the equations. 


Example 4.1.2: First, we consider salt and brine tanks, but this time water flows from one 
to the other and back. We again consider that the tanks are well-mixed. 


a) ee 


ee a, Ne 
Ly LQ 
oe 


Figure 4.1: A closed system of two brine tanks. 


Suppose we have two tanks, each containing volume V liters of salt brine. The amount of 
salt in the first tank is 7; grams, and the amount of salt in the second tank is x2 grams. The 
liquid is perfectly mixed and flows at the rate r liters per second out of each tank into the 
other. See Figure 4.1. 


Solution: The rate of change of x1, that is x, is the rate of salt coming in minus the rate 
going out. The rate coming in is the density of the salt in tank 2, that is 77, times the rate r. 
The rate coming out is the density of the salt in tank 1, that is +, times the rate r. In other 


words it is 
, v2 Uy r : is et ) 
Ly = r= r= LQ — t= v.— XM). 
V V 


V V V 
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Similarly we find the rate x25, where the roles of x; and x2 are reversed. All in all, the system 
of ODEs for this problem is 


(2 — £1), 
Ly = —(X1 — 22). 


In this system we cannot solve for x, or #2 separately. We must solve for both x; and x2 at 
once, which is intuitively clear since the amount of salt in one tank affects the amount in the 
other. We can’t know x, before we know 22, and vice versa. 

We don’t yet know how to find all the solutions, but intuitively we can at least find some 
solutions. Suppose we know that initially the tanks have the same amount of salt. That is, we 
have an initial condition such as 7(0) = 22(0) = C. Then clearly the amount of salt coming 
and out of each tank is the same, so the amounts are not changing. In other words, x7; = C 
and r2 = C (the constant functions) is a solution: x = x5 =0, and rg — 4, = 21 — 42 = 0, 
so the equations are satisfied. 

Let us think about the setup a little bit more without solving it. Suppose the initial 
conditions are 7;(0) = A and x2(0) = B, for two different constants A and B. Since no salt is 
coming in or out of this closed system, the total amount of salt is constant. That is, x, + x9 
is constant, and so it equals A+ B. Intuitively if A is bigger than B, then more salt will flow 
out of tank one than into it. Eventually, after a long time we would then expect the amount 
of salt in each tank to equalize. In other words, the solutions of both x; and x2 should tend 
towards 1, Once you know how to solve systems you will find out that this really is SO. | 


Example 4.1.3: Another example that showcases how systems work is different ways that 
populations of animals can interact. There are two main interactions that we will consider. 
The first of these is of two “competing species.” The idea here is that there are two species 
that are trying to coexist in a given area. On their own (without the other species), each one 
would grow exponentially, but any interaction between the two species is negative for both of 
them, because they share the types of food and other resources that they need to survive 
and grow. This gives rise to a system of differential equations of the form 


dx, ; 
— = ax, — 041 2X 
dt 1 142 
dx» d 
—— = CX2 — AX1XQ. 
dt 2 142 


In the system here, the coefficient a represents the growth rate of species 1 on it’s own, 
b represents the amount to which the competition for resources affects the growth rate of 
species 1, c represents the growth rate of species 2, and d represents the magnitude of how 
the competition affects the growth of species 2. This type of system can also be written to 
contain logistic growth terms for the two species, resulting in 


on 
dt 

ad 
dt 


= ax, (Ky = r1) = bx129 


= cx2(K = x2) = dx122. 
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The other main population model to consider is a “predator-prey” interaction. The key 
components of this model are that the prey population will grow on it’s own and the interaction 
between the two populations is negative, because the presence of predator population will 
cause the prey population to decrease. On the other hand, the predator population will die 
off on it’s own (without a food source) but the interaction with the prey population causes 
the predator population to increase. This gives rise to the system of differential equations 


dx h 
—=axr—br 
dt : 
dy 

7 =Cy any 


where x is the prey population and y is the predator population. We will take another look 
at both of these examples in § 5.3 once we have more terminology and techniques to discuss 
them. 


Example 4.1.4: Let us look at a second order example. We return to the mass and spring 
setup, but this time we consider two masses. 


Consider one spring with constant k and two masses ™, k 
and my. Think of the masses as carts that ride along a straight m1 NWA ms 
track with no friction. Let 7; be the displacement of the first e_e e_e 
cart and x2 be the displacement of the second cart. That is, as + 


we put the two carts somewhere with no tension on the spring, 
and we mark the position of the first and second cart and call those the zero positions. Then 
x; measures how far the first cart is from its zero position, and x2 measures how far the 
second cart is from its zero position. The force exerted by the spring on the first cart is 
k(xq — x1), since x2 — x; is how far the string is stretched (or compressed) from the rest 
position. The force exerted on the second cart is the opposite, thus the same thing with a 
negative sign. Newton’s second law states that force equals mass times acceleration. So the 
system of equations is 


mx, = k(r2— 21), 
MoX5 = —k(x = Z1). 


Again, we cannot solve for the x; or £2 variable separately. That we must solve for both 
x, and 2X at once is intuitively clear, since where the first cart goes depends on exactly where 
the second cart goes and vice versa. 


4.1.3 Changing to first order 


Before we talk about how to handle systems, let us note that in some sense we need only 
consider first order systems. Let us take an n™ order differential equation 


y™ = F(y*,...,y',y, 2) 


that we would like to convert into a first order system. To do this, we first consider what a 
first order system would look like. A first order system consists of a set of equations involving 
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the derivative of each of our variables. Let’s start with the first variable uj = y. What is 
the derivative of y? Well, it’s y’, and we don’t have a way to represent this in terms of our 
variables (u,) without any derivatives. So, we add a new variable wz that we define to be y’, 
which makes the first equation in our system ui), = ug. 

Well, now we have uz and we need to determine what its derivative is. Since uy = y/, 
us = y”. If the order of the equation n is 2, we then have an equation to define what y” is in 
terms of y’, y, and xz, which are uz, u;, and x in our new system. If that’s the case, we’re 
done, and if not, we need to define a new variable u3 = y” so that u4 = us. We can continue 
this process over and over again. 

When do we stop? As illustrated in the previous example with n = 2, we stop when our 
derivative u/, is the nth derivative of y. This works because our equation tells us exactly 
what y‘” is in terms of lower order terms, which we have already defined variables for. Thus, 


we define new variables uy, u2,...,Un and write the system 
/ —<_ 
Uy, = U2, 
/ 
Ug = U3, 
/ ——a 
Un-1 = Un; 
eer 
Ui =F Ug Wh as AG, 2 
We solve this system for u1, U2, ..., Un. Once we have solved for the u’s, we can discard ug 


through u,, and let y = u;. This y solves the original equation. 


Example 4.1.5: Take 2!” = 2x” + 82'+2a+t. Convert this equation into a first order 
system. 


Solution: Letting wu; = 7, ug = 2’, u3 = vw”, we find the system: 
Sy ty gs Us = 2u3 + Bug + ur +t. 


Since this is a linear system, we can also write this in matrix-vector form, which will be useful 
for systems that we will analyze later. To do this, we define a vector w as 


U1 
i= U2 
U3 
Then, we know that 
tt U2 
So U3 
Us 2u3 + 8u2 +u,+t 


We want to rewrite this equation using the vector w and a matrix. We can rewrite this last 
vector as 


Uy U2 0 
i = U3 ae Ae) 
Us uy + 8u2 + 2us t 
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and the right-hand side of this equation can be written as 
U2 0 0 1 0 0 
i = U3 + j;0}; = {0 O 1} w+ j0 
Uy + 8uy + 2u3 t 1 8 2 t 


(Verify that the matrix multiplication works out here!) Therefore, we can write this first 
order system as 


<2 


Note that if the equation above was non-linear, it would not be possible to write the 
system version in an appropriate matrix form. It is also important to know how to take 
initial conditions into account with these problems. 


Example 4.1.6: Convert the initial value problem 
gl" = detg"” — 3(a')? + t sin(x) + (#2 +1) et). =o. 20S =], 2°04 


into a system of first order equations. Simplify the expression as much as possible. 


Solution: We follow the same procedure as the previous example. We define variables uj, 
U2, UZ as 
CH= BeH7" aS" 


so that we have the differential equations 
Uy, = U2 Up=ug ug=2” = 4etug — 3u5 +f sin(u,) + (? +1) 


which we can write in vector form as 


Uy U2 
GS Nis |S U3 
us Ae'uz — 3u2 +t? sin(u,) + (t? + 1) 


We would now want to try to convert this into matrix form. However, the matrix that we 
come up with should not depend on wu at all. In this case, it would mean that we want to 
write this equation as 


* * *L LU 0 
“a’=|* * *! [uo] + 0 
* * *€| 1us eal 


since the extra term needs to be everything that does not depend on u. However, while we 
can determine the first two rows of the matrix, we can not determine the last row. There is 
no way to pick terms independent of u to fill in the three stars in the bottom row in order to 
make the bottom term in the matrix-vector product to equal 4e’us — 3u3 + t? sin(u:). The 
issue here is that the equation is non-linear; the v3 term and the sin(w:) term can not be 
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written in this way. Therefore, the best we can do is the vector form, and it can’t be written 
in matrix form. 
The last thing we need to deal with is the initial conditions. Since the conditions say that 


HOS? 0) =H) e0ye4 
and we have that uy = 27, ug =2', u3 = x”, this means that the initial condition should be 
uz(0) = 2: Uu2(0) = -1, u3(0) = A, 


or 


Thus, the full way to write this initial value problem in system form is 


U2 2 
u = UZ u(0) = |-1 
4e'us — 3u3 + t? sin(u,) + (t? + 1) 4 


A similar process can be followed for a system of higher order differential equations. For 
example, a system of k differential equations in k unknowns, all of order n, can be transformed 
into a first order system of n x k equations and n x k unknowns. 


Example 4.1.7: Consider the system from the carts example, 
mix} = k(x2— 1), Moxy = —k(x2—2)). 


Let wy = ©, Ug = v, Ug = 2, U4 = V4. The second order system becomes the first order 
system 


iL, =p, MyUy = k(ug — uy), = Us mau, = —k(ug — uy). 
Example 4.1.8: The idea works in reverse as well. Consider the system 
e=lty-r, y=, 


where the independent variable is t. We wish to solve for the initial conditions x(0) = 1, 
y(0) = 0. 

Solution: If we differentiate the second equation, we get y” = x’. We know what 2’ is in 
terms of x and y, and we know that x = y’. So, 


y =a =Q-—xr=2Qy-y’. 


We now have the equation y” + y' — 2y = 0. We know how to solve this equation and we find 
that y = Cye~** + Cye. Once we have y, we use the equation y’ = x to get x. 


g=yl = —2C;e "7 + Coe’. 
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We solve for the initial conditions 1 = x(0) = —2C; + C2 and 0 = y(0) = Ci + Cy. Hence, 
Cy = —C, and 1 = 3C,. So C) = —1/3 and C2 = 1/3. Our solution is 


Det hie® =p 7 gt 
1 — a SS. 
3 id 3 
| 


Exercise 4.1.1: Plug in and check that this really is the solution. 


It is useful to go back and forth between systems and higher order equations for other 
reasons. For example, software for solving ODE numerically (approximation) is generally 
for first order systems. So to use it, you have to take whatever ODE you want to solve and 
convert it to a first order system. In fact, it is not very hard to adapt computer code for 
the Euler or Runge-Kutta method for first order equations to handle first order systems. 
We essentially just treat the dependent variable not as a number but as a vector. In many 
mathematical computer languages there is almost no distinction in syntax. 


4.1.4 Autonomous systems and vector fields 


A system where the equations do not depend on the independent variable is called an 
autonomous system. For example the system y’ = 2y — x, y’ = x is autonomous as t is the 
independent variable but does not appear in the equations. 

For autonomous systems we can draw the so-called direction field or vector field, a plot 
similar to a slope field, but instead of giving a slope at each point, we give a direction (and 
a magnitude). The previous example, xz’ = 2y — x, y’ = x, says that at the point (x,y) the 
direction in which we should travel to satisfy the equations should be the direction of the 
vector (2y — x,x) with the speed equal to the magnitude of this vector. So we draw the 
vector (2y — x,x) at the point (x,y) and we do this for many points on the xry-plane. For 
example, at the point (1,2) we draw the vector (2(2) — 1,1) = (3, 1), a vector pointing to the 
right and a little bit up, while at the point (2,1) we draw the vector (2(1) — 2,2) = (0,2) a 
vector that points straight up. When drawing the vectors, we will scale down their size to fit 
many of them on the same direction field. We are mostly interested in their direction and 
relative size. See Figure 4.2 on the following page. 

We can draw a path of the solution in the plane. Suppose the solution is given by x = f(t), 
y = g(t). We pick an interval of t (say 0 < t < 2 for our example) and plot all the points 
(f(t), g(t)) for t in the selected range. The resulting picture is called the phase portrait (or 
phase plane portrait). The particular curve obtained is called the trajectory or solution curve. 
See an example plot in Figure 4.3 on the next page. In the figure the solution starts at 
(1,0) and travels along the vector field for a distance of 2 units of t. We solved this system 
precisely, so we compute x(2) and y(2) to find 7(2) » 2.475 and y(2) = 2.457. This point 
corresponds to the top right end of the plotted solution curve in the figure. 

Notice the similarity to the diagrams we drew for autonomous systems in one dimension. 
But note how much more complicated things become when we allow just one extra dimension. 

We can draw phase portraits and trajectories in the ry-plane even if the system is not 
autonomous. In this case however we cannot draw the direction field, since the field changes 
as t changes. For each t we would get a different direction field. 
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Figure 4.2: The direction field for x’ =2y—2, Figure 4.8: The direction field for x’ = 2y — 
y =o x, y’ = x with the trajectory of the solution 
starting at (1,0) forO0<t< 2. 


4.1.5 Picard’s theorem 


Perhaps before going further, let us mention that Picard’s theorem on existence and uniqueness 
still holds for systems of ODE. Let us restate this theorem in the setting of systems. A 
general first order system is of the form 


i Py (25s vk ws Bs 
BS Io (a, 1550-54 bs 

(4.1) 
Dy is Pies ak) 


Theorem 4.1.1 (Picard’s theorem on existence and uniqueness for systems) 


Iftor every 7 — 1) 2.22.57 and every f — Io). cach Fis contimuous and the 
0 ».0 


Sn Glas ee : : : 
derivative an exists and is continuous near some (}, X5,... eon i”), then a solution to 


(4.1) subject to the mtialcondition a4(¢)i— «1, . tot \— 2, et) — 7 exists 


(at least for some small interval of t’s) and is unique. 


That is, a unique solution exists for any initial condition given that the system is reasonable 
(Ff; and its partial derivatives in the x variables are continuous). As for single equations we 
may not have a solution for all time t, but at least for some short period of time. 

As we can change any nth order ODE into a first order system, then we notice that this 


theorem provides also the existence and uniqueness of solutions for higher order equations 
that we have until now not stated explicitly. 
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4.1.6 Exercises 


Exercise 4.1.2: Verify that 1;(t) = 2e~' — 2e~*, xo(t) = e~' — 2e~*' solves the system 
Li = —2x2, Bp = L1 — 3x2. 


Exercise 4.1.3: Verify that x(t) = —2te~*! —2e~**, xo(t) = 2te~** + 3e~* solves the system 
vy = —54, — Ino, Lh = 221 — Zo. 


Exercise 4.1.4: Find the general solution of x, = x2 — 2% + t, v4 = 22. 

Exercise 4.1.5: Find the general solution of «/, = 3x, — 2+ e', 74 = 2). 
Exercise 4.1.6:* Find the general solution to y = 3y1, ys = y1 + Y2, ¥3 = Yi + Y3- 
Exercise 4.1.7:* Solve y' = 2x, vc =x+y, x(0) =1, y(0) =3. 

Exercise 4.1.8: Write ay” + by’ + cy = f(x) as a first order system of ODEs. 


Exercise 4.1.9: Write x” + y?y' — 2° = sin(t), y"+(2' + y/)? — = 0 as a first order system 
of ODEs. 


Exercise 4.1.10:* Write x” =x +t as a first order system. 
Exercise 4.1.11:* Write y/ +y,+ yo =t, yg + y1 — yo = as a first order system. 
Exercise 4.1.12: Write y — t?y” + ety’ — (2t+ 1)y = cos(t) as a first order system. 
Exercise 4.1.13: Write the initial value problem 

y’ —2ry'+3y=sin(r) y(0)=1, y/(0) =-2 


as an initial value problem for a first order system of ODEs. Make sure to indicate how the 
initial condition appears as a part of this problem. 


Exercise 4.1.14: Write the initial value problem 
yf —(y+1)’y —e% =cos(z) —-y(0) = -1, (0) =5 


as an initial value problem for a first order system of ODEs. Make sure to indicate how the 
initial condition appears as a part of this problem. Can this be written in matrix form? Why 
or why not? 


Exercise 4.1.15: Write the initial value problem 


y + ety” —4cos(x)y! + (a? + Ly = y(0) = 2, y'(0) = -3, y”(0) =0, y(0) =1 


ees 
as an initial value problem for a first order system of ODEs. Make sure to indicate how the 
initial condition appears as a part of this problem. 


Exercise 4.1.16: Suppose two masses on carts on frictionless surface are at displacements 
XZ, and x2 as in Example 4.1.4 on page 273. Suppose that a rocket applies force F' in the 
positive direction on cart x1. Set up the system of equations. 
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Exercise 4.1.17:* Suppose two masses on carts on frictionless surface are at displacements 
x1 and £2 as in Example 4.1.4 on page 273. Suppose initial displacement is x1(0) = x2(0) = 0, 
and initial velocity is x'(0) = x4(0) =a for some number a. Use your intuition to solve the 
system, explain your reasoning. 


Exercise 4.1.18: Suppose the tanks are as in Example 4.1.2 on page 271, starting both at 
volume V, but now the rate of flow from tank 1 to tank 2 is r,, and rate of flow from tank 
2 to tank one is rg. In particular, the volumes will now be changing. Set up the system of 
equations. 


Exercise 4.1.19:* Suppose the tanks are as in Example 4.1.2 on page 271 except that clean 
water flows in at the rate s liters per second into tank 1, and brine flows out of tank 2 and 
into the sewer also at the rate of s liters per second. 


a) Draw the picture. 
b) Set up the system of equations. 
c) Intuitively, what happens as t goes to infinity, explain. 


Exercise 4.1.20: Match the systems of differential equations below to their corresponding 
slope fields. Justify. 


3 =e Sa 


(11) 


b) 


Exercise 4.1.21: Match the systems of differential equations below to their corresponding 
slope fields. Justify. 


dx dx 2 dx 
Sg See . JG =x 5 | ge = yt2 
(1) 9% (rae (ise * 

ae ee a ty a —etytil 
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4.2 Matrices and linear systems 


Attribution: [JL], §3.2. 


After this section, you will be able to: 


e Define and perform addition and multiplication operations on matrices, 
e Compute the determinant of a square matrix, and 


e Find eigenvalues and eigenvectors of a square matrix. 


This section is meant to summarize the parts of linear algebra that will be necessary in 
the process of developing and solving linear systems of differential equations. All of this 
information is covered in more detail in Chapter 3, so you can find more information there. 
If you went through that chapter already, this section will serve as a review. 


4.2.1 Matrices and vectors 


Before we start talking about linear systems of ODEs, we need to talk about matrices, so let 
us review these briefly. A matriz is an m x n array of numbers (m rows and n columns). For 
example, we denote a 3 x 5 matrix as follows 


G11 G12 413 Aig A415 
A= Q21 G22 423 G24 425 
431 432 433 434 435 


The numbers a;; are called elements or entries. 

By a vector we usually mean a column vector, that is an m x 1 matrix. If we mean a row 
vector, we will explicitly say so (a row vector is a 1 x n matrix). We usually denote matrices 
by upper case letters and vectors by lower case letters with an arrow such as 7 or b. By 0 we 
mean the vector of all zeros. 

We define some operations on matrices. We want 1 x 1 matrices to really act like numbers, 
so our operations have to be compatible with this viewpoint. 

First, we can multiply a matrix by a scalar (a number). We simply multiply each entry in 
the matrix by the scalar. For example, 


»[i 2 3)_[2 4 6 
40.5 °6)|~ |S a 121° 
Matrix addition is also easy. We add matrices element by element. For example, 
1 2 3 4 1 1-1) |2 3 2 
4 5 6 02: 4.)°-|4 7 10)" 


If the sizes do not match, then addition is not defined. 
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If we denote by 0 the matrix with all zero entries, by c, d scalars, and by A, B, C matrices, 
we have the following familiar rules: 


ALKA AA, 
ARBEBEA 
(A&B) ECS ALB AC), 
c(A+B)=cA+cB, 
(c+ d)A=cA+dA. 


Another useful operation for matrices is the so-called transpose. This operation just swaps 
rows and columns of a matrix. The transpose of A is denoted by A’. Example: 


R24 - ae 
4 5 6 aan 


4.2.2 Matrix multiplication 


Let us now define matrix multiplication. First we define the so-called dot product (or inner 
product) of two vectors. Usually this will be a row vector multiplied with a column vector of 
the same size. For the dot product we multiply each pair of entries from the first and the 
second vector and we sum these products. The result is a single number. For example, 


[a1 a2 as| » | bo] = ab; + agby + agbs. 


And similarly for larger (or smaller) vectors. 

Armed with the dot product we define the product of matrices. First let us denote by 
row;(A) the i** row of A and by column,(A) the j column of A. For an m x n matrix A 
and an n xX p matrix B we can define the product AB. We let AB be an m x p matrix whose 
ij*® entry is the dot product 

row;(A) - column,(B). 


Do note how the sizes match up: m x n multiplied by n x p is m x p. Example: 


—_a 
Re Re 


0) ea eee 
6-0] |15 5 1]° 


For multiplication we want an analogue of a 1. This analogue is the so-called identity 
matrix. The identity matrix is a square matrix with 1s on the diagonal and zeros everywhere 
else. It is usually denoted by J. For each size we have a different identity matrix and so 


on 
eR 
a 
| 
re 
ie th 
on 
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sometimes we may denote the size as a subscript. For example, the Jz; would be the 3 x 3 
identity matrix 


We have the following rules for matrix multiplication. Suppose that A, B, C’ are matrices 
of the correct sizes so that the following make sense. Let a denote a scalar (number). 


A(BC) = (AB)C, 
A(B+C)=AB+AC, 
(B+C)A= BA+CA, 

a(AB) = (aA)B = A(aB), 

IA=A=AI. 


A few warnings are in order. 


(i) AB ¥ BA in general (it may be true by fluke sometimes). That is, matrices do not 
commute. For example, take A = [{}] and B= [$9]. 


(ii) AB = AC does not necessarily imply B = C, even if A is not 0. 


(iii) AB = 0 does not necessarily mean that A = 0 or B= 0. Try, for example, A = B = 
[oo]: 
For the last two items to hold we would need to “divide” by a matrix. This is where the 
matrix inverse comes in. Suppose that A and B are n x n matrices such that 


AB=I=BA. 


Then we call B the inverse of A and we denote B by A™'. If the inverse of A exists, then we 
call A invertible. If A is not invertible, we sometimes say A is singular. 

If A is invertible, then AB = AC does imply that B = C (in particular the inverse of 
A is unique). We just multiply both sides by A! (on the left) to get AT‘AB = A~'AC or 
IB =IC or B=C. We can also see from the definition that (A~!)~* = A. 


4.2.3. The determinant 


For square matrices we define a useful quantity called the determinant. We define the 
determinant of a 1 x 1 matrix as the value of its only entry. For a 2 x 2 matrix we define 


det ( il) id be, 
cd 


Before trying to define the determinant for larger matrices, let us note the meaning of the 
determinant. Consider an n x n matrix as a mapping of the n-dimensional euclidean space R” 
to itself, where x gets sent to Ax. In particular, a 2 x 2 matrix A is a mapping of the plane 
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to itself. The determinant of A is the factor by which the area of objects changes. If we take 
the unit square (square of side 1) in the plane, then A takes the square to a parallelogram of 
area |det(A)|. The sign of det(A) denotes changing of orientation (negative if the axes get 


flipped). For example, let 
1 1 
a le i : 


Then det(A) = 1+ 1 = 2. Let us see where the (unit) square with vertices (0,0), (1,0), (0,1), 
and (1,1) gets sent. Clearly (0,0) gets sent to (0,0). 

t. AV) 

—1 1} }1] Jo} 


a bl-E) Gdil-b): 


The image of the square is another square with vertices (0,0), (1,—1), (1,1), and (2,0). The 
image square has a side of length V2 and is therefore of area 2. 

If you think back to high school geometry, you may have seen a formula for computing the 
area of a parallelogram with vertices (0,0), (a,c), (b,d) and (a+b,c+d). And it is precisely 


(ED: 


The vertical lines above mean absolute value. The matrix [%%] carries the unit square to the 
given parallelogram. 


Let us look at the determinant for larger matrices. We define A;; as the matrix A with 
the i** row and the j column deleted. To compute the determinant of a matrix, pick one 
row, say the i** row and compute: 


det(A) = s (—1)' a,j det(Aj,). 


j=l 


For the first row we get 


+din det(Ai,) if n is odd, 


det(A) = det(Aj1) — ay det(Aj2) + a,3 det(Aj3) —--- 
A) = = Bede e ay aise teenie if n even. 


We alternately add and subtract the determinants of the submatrices A;; multiplied by 
a;; for a fixed 7 and all j. For a 3 x 3 matrix, picking the first row, we get det(A) = 
Qy1 det(A11) — aj2 det (Aj2) + a13 det (Aj3). For example, 


1 2 3 
det 4 5 6 =1-aet (|? 5) ~2-aee ( [7 5) +3-aet ( [2 4) 
8 9 7 9 7 8 
7 8 9 
=1(5-9-—6-8) —2(4-9-6-7)+3(4-8—5-7) =0. 


The numbers (—1)'*? det(Aj;) are called cofactors of the matrix and this way of computing 
the determinant is called the cofactor expansion. No matter which row you pick, you always 
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get the same number. It is also possible to compute the determinant by expanding along 
columns (picking a column instead of a row above). It is true that det(A) = det(A”). 
A common notation for the determinant is a pair of vertical lines: 


:4-m(( 3) 


I personally find this notation confusing as vertical lines usually mean a positive quantity, 
while determinants can be negative. Also think about how to write the absolute value of a 
determinant. I will not use this notation in this book. 


Think of the determinants telling you the scaling of a mapping. If B doubles the sizes of 
geometric objects and A triples them, then AB (which applies B to an object and then A) 
should make size go up by a factor of 6. This is true in general: 


det(AB) = det(A) det(B). 


This property is one of the most useful, and it is employed often to actually compute 
determinants. A particularly interesting consequence is to note what it means for existence 
of inverses. Take A and B to be inverses of each other, that is AB = J. Then 


det(A) det(B) = det(AB) = det(Z) = 1. 


Neither det(A) nor det(B) can be zero. Let us state this as a theorem as it will be very 
important in the context of this course. 


Theorem 4.2.1 


An n X n matrix A is invertible if and only if det(A) 4 0. 


In fact, det(A~') det(A) = 1 says that det(A~+) = aa So we even know what the 


determinant of A~! is before we know how to compute A™t. 
There is a simple formula for the inverse of a 2 x 2 matrix 


ab) 1 [dad -b 
c d| ad—be|—c al’ 


Notice the determinant of the matrix [¢ 5] in the denominator of the fraction. The formula 


only works if the determinant is nonzero, otherwise we are dividing by zero. 


4.2.4 Solving linear systems 


One application of matrices we will need is to solve systems of linear equations. This is best 
shown by example. Suppose that we have the following system of linear equations 


2241 + 229 + 223 = 2, 


Uy+ +373 =5, 
%1+4¢2+ x3 = 10. 
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Without changing the solution, we could swap equations in this system, we could multiply 
any of the equations by a nonzero number, and we could add a multiple of one equation to 
another equation. It turns out these operations always suffice to find a solution. 

It is easier to write the system as a matrix equation. The system above can be written as 


2 2 2) [xy y) 
1 1 3 LQ) = 5 
1 4 1) [23 10 


To solve the system we put the coefficient matrix (the matrix on the left-hand side of the 
equation) together with the vector on the right and side and get the so-called augmented 
matrix 


22s 2:2 
5 Tea ames Ue 
141 
We apply the following three elementary operations. 
(i) Swap two rows. 
(ii) Multiply a row by a nonzero number. 


(iii) Add a multiple of one row to another row. 


We keep doing these operations until we get into a state where it is easy to read off the 
answer, or until we get into a contradiction indicating no solution, for example if we come up 
with an equation such as 0 = 1. 

Let us work through the example. First multiply the first row by !/2 to obtain 


yeaa Cee SO 
yaa Vinee Ne 
14 1/1 


oS 


Now subtract the first row from the second and third row. 
11 éi1i)i1 
00 2/4 
0 3 0/9 


Multiply the last row by 1/3 and the second row by 1/2. 


1 1 i1yj)i1 

00 1)]2 

OL Oye 
Swap rows 2 and 3. 

1 1i1j)i1 

0 1 3 

00 1)]2 
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Subtract the last row from the first, then subtract the second row from the first. 


1 0 0| -4 
Oaks.) 23 
00 1) 2 
If we think about what equations this augmented matrix represents, we see that 7; = —4, 


2 = 3, and x3 = 2. We try this solution in the original system and, voila, it works! 
Exercise 4.2.1: Check that the solution above really solves the given equations. 
We write this equation in matrix notation as 
AZ = b, 


where A is the matrix E i 3] and b is the vector | 3: The solution can also be computed 
via the inverse, 


#— AAZ = AW. 


It is possible that the solution is not unique, or that no solution exists. It is easy to tell if 
a solution does not exist. If during the row reduction you come up with a row where all the 
entries except the last one are zero (the last entry in a row corresponds to the right-hand 
side of the equation), then the system is inconsistent and has no solution. For example, for 
a system of 3 equations and 3 unknowns, if you find a row such as [0 0 0 | 1] in the 
augmented matrix, you know the system is inconsistent. That row corresponds to 0 = 1. 


You generally try to use row operations until the following conditions are satisfied. The 
first (from the left) nonzero entry in each row is called the leading entry. 


(i) The leading entry in any row is strictly to the right of the leading entry of the row 
above. 


(ii) Any zero rows are below all the nonzero rows. 
(iii) All leading entries are 1. 
(iv) All the entries above and below a leading entry are zero. 


Such a matrix is said to be in reduced row echelon form. The variables corresponding to 
columns with no leading entries are said to be free variables. Free variables mean that we 
can pick those variables to be anything we want and then solve for the rest of the unknowns. 


Example 4.2.1: The following augmented matrix is in reduced row echelon form. 
ile Olle 
00 1}1 
0 0 0|0 


Suppose the variables are 71, %2, and 73. Then 22 is the free variable, x71 = 3 — 2x2, and 
L3 = 1. 
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On the other hand if during the row reduction process you come up with the matrix 


1 2 13)3 
OF $0 Seiad fc, 
Os OF O58 

there is no need to go further. The last row corresponds to the equation 02%; + 0x2 + 0x3 = 3, 
which is preposterous. Hence, no solution exists. 


4.2.5 Computing the inverse 


If the matrix A is square and there exists a unique solution % to Av = b for any b (there 
are no free variables), then A is invertible. Multiplying both sides by A~', you can see that 
7 = A-!b. So it is useful to compute the inverse if you want to solve the equation for many 
different right-hand sides b. 

We have a formula for the 2 x 2 inverse, but it is also not hard to compute inverses of 
larger matrices. While we will not have too much occasion to compute inverses for larger 
matrices than 2 x 2 by hand, let us touch on how to do it. Finding the inverse of A is actually 
just solving a bunch of linear equations. If we can solve Ax, = €; where €; is the vector with 
all zeros except a 1 at the k* position, then the inverse is the matrix with the columns Z,; 
for k = 1,2,...,n (exercise: why?). Therefore, to find the inverse we write a larger n x 2n 
augmented matrix | A | I], where J is the identity matrix. We then perform row reduction. 
The reduced row echelon form of [A | J] will be of the form [J | A~'] if and only if A is 
invertible. We then just read off the inverse A7?. 


4.2.6 Eigenvalues and eigenvectors of a matrix 


Let A be a constant square matrix. Suppose there is a scalar \ and a nonzero vector v such 
that 

AU = WW. 
We call \ an eigenvalue of A and we call v a corresponding eigenvector. 


Example 4.2.2: The matrix [21] has an eigenvalue A = 2 with a corresponding eigenvector 


- e \E)-E-2f 


Let us see how to compute eigenvalues for any matrix. Rewrite the equation for an 
eigenvalue as 
(A—dAD¢=0. 
This equation has a nonzero solution v only if A — AJ is not invertible. Were it invertible, we 
could write (A — \J)~'(A — AD)@ = (A — AI)~'0, which implies ¢ = 0. Therefore, A has the 
eigenvalue \ if and only if ’ solves the equation 


det(A — XI) =0. 
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Consequently, we will be able to find an eigenvalue of A without finding a corresponding 
eigenvector. An eigenvector will have to be found later, once \ is known. 


Example 4.2.3: Find all eigenvalues of E 2 | ; 


Solution: We write 


211 1 0 0 2—X 1 1 
det | }1 2 O} —A]JO 1 O = det 1 2-XA 0 = 
0: Ds 2 Os, OO al 0 0 2-A 
= (2—A)((2— A)? = 1) = -(A- YO — 2)(A- 3). 
So the eigenvalues are \ = 1, A = 2, and A = 3. | 


For an n xX n matrix, the polynomial we get by computing det(A — AJ) is of degree n, and 
hence in general, we have n eigenvalues. Some may be repeated, some may be complex. 


To find an eigenvector corresponding to an eigenvalue A, we write 
(A— Ale = 0, 


and solve for a nontrivial (nonzero) vector v. If A is an eigenvalue, there will be at least one 
free variable, and so for each distinct eigenvalue 4, we can always find an eigenvector. 


Example 4.2.4: Find an eigenvector of E 2 0 corresponding to the eigenvalue A = 3. 


Solution: We write 


2A 10 0]\ fu -1 1 1) fu 
(A-ADv={/1 2 0] —3]0 1 Of | Jeo} =} 1 -1 Of Je] =o. 
mene 00 1]/ Jos 0 0 -1] Jos 


=p, 2, Ae 
We TOU 4 
0 0 -1]0 


and perform row operations (exercise: which ones?) until we get: 


1 
0 
0 


SSS: 


The entries of V have to satisfy the equations vj — vg = 0, v3 = 0, and vz is a free variable. 
We can pick v2 to be arbitrary (but nonzero), let vj = v2, and of course v3 = 0. For example, 


if we pick vo = 1, then v = fai Let us verify that v really is an eigenvector corresponding 
10 A= 3) 


Yay! It worked. | 
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Exercise 4.2.2 (easy): Are eigenvectors unique? Can you find a different eigenvector for 
A = 3 in the example above? How are the two eigenvectors related? 


Exercise 4.2.3: When the matrix is 2 x 2 you do not need to do row operations when 


computing an eigenvector, you can read it off from A — XI (if you have computed the 


eigenvalues correctly). Can you see why? Explain. Try it for the matrix [? 3]. 


4.2.7 Exercises 


Exercise 4.2.4: Let A and B be the matrices below. 


1 4 -1 Ore 2 38 
Ava 12-02 «3 B= |l —4 -2 
1 -—2 3 25, a 


Compute A+ 3B, AB, and BA. 


Exercise 4.2.5: Solve [$7|Z= [2] by using matrix inverse. 


296 
Exercise 4.2.6: Compute determinant of FE 3 5 |. 


Exercise 4.2.7:* Compute determinant of [3 3 35] 


Poi, 
Exercise 4.2.8: Compute determinant of EH : 0 . Hint: Expand along the proper row or 


column to make the calculations simpler. 


Exercise 4.2.10: For which h is E 
several? Infinitely many? 


Coon 


Exercise 4.2.11:* Find t such that | |, $] is not invertible. 
Exercise 4.2.12: For which h is [6 h 6 not invertible? Find all such h. 


Exercise 4.2.13:* Solve the system of equations 
Avy = 22 le 473 = —8 
Ly = 323 =12 
—4Axr, + AX ahr 4x3 = -—8 


or determine that no solution exists. 
Exercise 4.2.14:* Solve the system of equations 
—%X, — 4%. + 243 = 11 
321 — 8%. + 43 = 13 


—521 = DL a 3X3 =9 


or determine that no solution exists. 


4.2. MATRICES AND LINEAR SYSTEMS 291 


Exercise 4.2.15:* Solve the system of equations 
ey 3X9 = 323 = ] 
—321 = Ar + 4x3 =—3 
4a zoe 7X9 cae 7X3 aii 


or determine that no solution exists. 
Exercise 4.2.16:* Solve the system of equations 


Uy+ 3X9 —-13= 5 
224 +%2= —3 
—32, — 4r + 223 = —6 
or determine that no solution exists. 


Exercise 4.2.17: Solve E os 6. t= H : 
10 —2 3 


Exercise 4.2.18: Solve E 4 i| a 3]. 
633 0 

Exercise 4.2.19:* Solve [j 1,]% = [38]. 
3230 2 

Exercise 4.2.20: Solve [ a i c= i 
2343 1 


Exercise 4.2.21: Find 3 nonzero 2 x 2 matrices A, B, and C such that AB = AC but 
BEC. 


Exercise 4.2.22:* Suppose a,b,c are nonzero numbers. Let M = [6°], N= E b 0). 


a) Compute M~'. b) Compute N~'. 


Exercise 4.2.23 (easy): Let A be a3 x3 matrix with an eigenvalue of 3 and a corresponding 


eigenvector U = Ea . Find Av. 

Exercise 4.2.24:* Find the eigenvalues and eigenvectors for the matrix 
0 —2 
ces ca 

Exercise 4.2.25:* Find the eigenvalues and eigenvectors for the matrix 
—8 -—5 
8 4]- 

Exercise 4.2.26:* Find the eigenvalues and eigenvectors for the matrix 


t =o 7 
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4.3 Linear systems of ODEs 


Attribution: [JL], §3.8. 


After this section, you will be able to: 


e Use proper terminiology when discussing linear systems of differential equations 


and their solutions, 

e Determine whether a set of functions is linearly independent, and 

e Understand how the theory of non-homogeneous linear systems relates to the 
theory of non-linear equations. 


In order to get into the details of how to talk about and deal with linear systems of 
differential equations, we first need to talk about matrix- or vector-valued functions. Such 
a function is just a matrix or vector whose entries depend on some variable. If t is the 
independent variable, we write a vector-valued function X(t) as 


x(t) 
Ca ea 
alt) 
Similarly a matrie-valued function A(t) is 
ayi(t) ai2(t) +++ ain(t) 
A= on a) - an(t) 
He Oy Gal) <i> tae 


As long as the addition of vectors is defined, we can add vector-valued functions, and as 
long as the addition and multiplication of matrices are defined (like if they are the right size) 
we can multiply matrix-valued functions. In addition, the derivative A’(t) or ut is just the 
matrix-valued function whose ij‘ entry is a;;(t). We used this idea previously when talking 
about how to write first order systems from higher order equations in § 4.1. 

Rules of differentiation of matrix-valued functions are similar to rules for normal functions. 
Let A(t) and B(t) be matrix-valued functions. Let c a scalar and let C be a constant matrix. 
Then 


(A(t) + Bt) = Ad) + BO), 
(A(t) B(t))’ = 4’() Bt) + AV) BA), 
(cA(t))’ = cA'(A), 
(CA(t))’ = CA’), 
(AQ) C)' = A’(AC. 
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Note the order of the multiplication in the last two expressions because matrix multiplication 
is not commutative. 
A first order linear system of ODEs is a system that can be written as the vector equation 


=> 


H(t) = P(t)z(t) + F(t), 


> 


where P(t) is a matrix-valued function, and Z(t) and f(t) are vector-valued functions. We 
will often suppress the dependence on ¢t and only write 2” = Px + . A solution of the system 
is a vector-valued function # satisfying the vector equation. 

For example, the equations 


gv = 2ta,+ ex. + t?, 


can be written as ; 
2 fe t 
— > 
C= Fe | r+ 4 : 
We will mostly concentrate on equations that are not just linear, but are in fact constant 
coefficient equations. That is, the matrix P will be constant; it will not depend on t. 


When f =0 (the zero vector), then we say the system is homogeneous. For homogeneous 
linear systems we have the principle of superposition, just like for single homogeneous 
equations. 


Theorem 4.3.1 (Superposition) 


Let z’ = Pz be a linear homogeneous system of ODEs. Suppose that 71, Z2,..., 2, are 
n solutions of the equation and c;,¢2,...,C, are any constants, then 


= Cy X41 + Cok ap 888 Se Ch lias (42) 


is also a solution. Furthermore, if this is a system of n equations (P is n x n), and 
£1, £2,...,£, are linearly independent, then every solution Z can be written as (4.2). 


Linear independence for vector-valued functions is the same idea as for normal functions. 
The vector-valued functions 71, 72,...,%n are linearly independent if the only way to satisfy 
the equation 

CyB + co®y +++ + nF, = 0 


is by choosing the parameters c; = Co = --- = C, = 0, where the equation must hold for all t. 


Example 4.3.1: Determine if the sets S$, = {#1 = |? |, is = Ear t= ||} and 
i {#1 = BE to = ie £3 = eal, are linearly independent. 


Solution: The vector functions in 5S; are linearly dependent because %; + 73 = Xo, and this 
holds for all t. So c; = 1, co = —1, and c3 = 1 above will work. 
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On the other hand, the vector functions in S2 are linearly independent, even though this 
is only a slight change from 5). First write c,%1 + co%2 + c3%3 = 0 and note that it has to 
hold for all t. We get that 


: cyt? — c3t? 0 
CyL1 + Co%g + C3L3 = ey Cot ig - 7 A , 


In other words c,t? — c3t? = 0 and cit +cot+c3 = 0. If we set t = 0, then the second equation 
becomes cz = 0. But then the first equation becomes c,t? = 0 for all t and so c; = 0. Thus 
the second equation is just cot = 0, which means cp = 0. So cy = co = c3 = O is the only 
solution and £1, #2, and 23 are linearly independent. | 


The linear combination c,%1 + Co%_ + +--+ Cy, could always be also as 


where X(t) is the matrix with columns #1, @2,...,£,, and Gis the column vector with entries 
C1, C2,--+,Cn-. This is similar to the way that we could write linear combinations of vectors 
by putting them into a matrix, including how we talked about rank in § 3.4. Assuming 
that 21, %2,...,2Zp are linearly independent and solutions to a given system of differential 
equations, the matrix-valued function X(t) is called a fundamental matriz, or a fundamental 
matrix solution. 


To solve nonhomogeneous first order linear systems, we use the same technique as we 
applied to solve single linear nonhomogeneous equations. 


Theorem 4.3.2 


Let @ = P#+ e be a linear system of ODEs. Suppose 7, is one particular solution. 
Then every solution can be written as 


#= 2+ Zp, 


=> 


where Z, is a solution to the associated homogeneous equation (z” = PZ) 


The procedure for systems is the same as for single equations. We find a particular 
solution to the nonhomogeneous equation, then we find the general solution to the associated 
homogeneous equation, and finally we add the two together. 


Alright, suppose you have found the general solution of 2” = Px + f. Next suppose you 
are given an initial condition of the form 


(to) =b 


for some fixed ty and a constant vector b. Let X (t) be a fundamental matrix solution of the 
associated homogeneous equation (i.e. columns of X(t) are solutions). The general solution 
can be written as 

x(t) = X(t) E+ Z,(t). 
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We are seeking a vector ¢ such that 
b = £(to) = X (to) f+ Fp(to). 
In other words, we are solving for Cin the nonhomogeneous system of linear equations 
X (to) f= b — Z,(to). 
Example 4.3.2: In § 4.1 we solved the system 


tL, =i, 
t= 21 — 29, 
with initial conditions 7,(0) = 1, x2(0) = 2. Let us consider this problem in the language of 


this section. 
The system is homogeneous, so f(t) = 0. We write the system and the initial conditions 


z=[) Sle #0) = [3]. 


t 


as 


+ ee, 


We found the general solution is 7; = cye’ and x2 = rae Letting c; = 1 and 
co = 0, we obtain the solution cafe | Letting c; = 0 and co = 1, we obtain ee iF These 
two solutions are linearly independent, as can be seen by setting t = 0, and noting that the 
resulting constant vectors are linearly independent. In matrix notation, a fundamental matrix 


solution is, therefore, 


or in other words, 


A single elementary row operation shows ¢ = [ ara |e Our solution is 


co=n0e=[f OME hesied 


This new solution agrees with our previous solution from § 4.1. 


4.3.1 Exercises 


Exercise 4.3.1: Write the system x, = 2x7, — 3tr2 + sint, x, = e'v, + 3x2 + cost in the 


> 


form # = P(t)¢ + f(t). 
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Exercise 4.3.2:* Write x’ = 3x —y+e', y’ = tz in matrix notation. 
Exercise 4.3.3: Consider the third order differential equation 
y” +(y +1)? =e +sin(t +1). 


Convert this to a first order system and simplify as much as possible. Can you write this in 
the form < = AZ + f? Why or why not? 


Exercise 4.3.4: 
a) Verify that the system x’ = [}3]Z has the two solutions [+] e“’ and [ +,]e~*. 


b) Write down the general solution. 


c) Write down the general solution in the form x; =?, v2 =? (i.e. write down a formula 
for each element of the solution). 


Exercise 4.3.5: Verify that [4] e’ and [1,|e‘ are linearly independent. Hint: Just plug in 
t=0. 


Exercise 4.3.6:* Are eal and ee) linearly independent? Justify. 
Exercise 4.3.7: Verify that H e' and B e’ and B e** are linearly independent. Hint: 
You must be a bit more tricky than in the previous exercises. 
Exercise 4.3.8:* Are [rr ale and [er"] linearly independent? Justify. 
Exercise 4.3.9: Verify that [,;] and [‘] are linearly independent. 
Exercise 4.3.10: Take the system x +25 = %, v,— © = 2p. 
a) Write it in the form Ax’ = Bz for matrices A and B. 
b) Compute A~' and use that to write the system in the form x = PZ. 
Exercise 4.3.11:* 


a) Write x', = 2tx2, x = 2tx2 in matrix notation. 


b) Solve and write the solution in matrix notation. 
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4.4 Eigenvalue method 


Attribution: [JL], §3.4, 3.7. 


After this section, you will be able to: 


e Use the eigenvalue method to find straight-line solutions to constant-coefficient 
first order systems of ODE, 
e Find general solutions to systems with real and distinct eigenvalues, and 


e Solve initial value problems from all of these cases once the general solution has 
been found. 


In this section we will learn how to solve linear homogeneous constant coefficient systems 
of ODEs by the eigenvalue method. Suppose we have such a system 


where P is a constant square matrix. We wish to adapt the method for the single constant 
coefficient equation by trying the function e*’. However, # is a vector. So we try # = ve, 
where UV is an arbitrary constant vector. We plug this # into the equation to get 


Ave™ = Pe. 
Tr OT 


x! Px 


We divide by e* and notice that we are looking for a scalar \ and a vector @ that satisfy the 
equation 
Ad = PU. 

This means that we are looking for an eigenvalue \ with corresponding eigenvector UV 
for the matrix P. When we can find these, we will get solutions to the original system of 
differential equations of the form 

#(t) = ve. 
We get the easiest route to solutions when the matrix P has all real eigenvalues and the 
eigenvalues are all distinct, and can extend to deal with the complications that arise from 
complex and repeated eigenvalues. 

Another way to view these types of solutions are as “straight-line solutions.” A system of 


differential equations of the form 
f= Px, 


is an autonomous system of differential equations, because there is no explicit dependence 
on t on the right-hand side. When we solved autonomous equations in § 1.7, we started by 
looking for equilibrium solutions and built up from there. In this particular case, we are 
looking for vectors Z so that Px = 0. As long as P is invertible, the only vector that satisfies 
this is # = 0. So, that’s not super interesting, and doesn’t really tell us too much about the 
solution to the problem. 
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The next more involved type of solution 
we could look for is a straight-line solution. 
The idea is that this solution will either 
move directly (in a straight-line) towards or 
away from the origin. In the first order au- 
tonomous equation case, all of our solutions 
did this; they either moved towards or away 
from these equilibrium solutions. This may 
not be the case for systems, but we can try to 
find them. If a solution is going to move di- 
rectly towards or away from the origin, then 
the direction of change for the solution must | Figure 4.4: Position vector and possible direc- 
be parallel to the position vector. In Fig- | tion vectors for straight line solutions. 
ure 4.4, the vectors that point in the same 
or opposite direction of x will give rise to a straight-line solution, but vectors that do not 
point in this direction will give solutions that do not follow a straight-line through the origin. 

This criterion means that we need to have 


a 


f= AL 
for some constant 2. If this is the case, then we have 
Pea he 
and this is the equation for eigenvalues and eigenvectors of P. We are back to the same type 
of solution that we found previously. 
4.4.1 The eigenvalue method with distinct real eigenvalues 


OK. We have the system of equations 


of = PE. 
We find the eigenvalues 1, A2, ..., An of the matrix P, and corresponding eigenvectors vj, 
U2, ..., Un. Now we notice that the functions v,e*", v,e%2", ..., d,,e%* are solutions of the 


homogeneous system of equations and hence Z = cv ,e*! + cotge*2! +--+ + cntner! is a 
solution by superposition. 


Theorem 4.4.1 


Take x’ = Pz. If P is an n xX n constant matrix that has n distinct real eigenvalues ), 
2, ---, An, then there exist n linearly independent corresponding eigenvectors VU}, Uo, 
..., Un, and the general solution to #” = P# can be written as 


Ant 


i= cv e" =F CoV?! qr 68 0 Se CrUn€ 


The corresponding fundamental matrix solution is 


X(t) = [wer Toe... wwe |; 
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That is, X(t) is the matrix whose j‘" column is vje*". 
Example 4.4.1: Consider the system 
2.1. 
p= [LB 0 le 
0 0 2 


Find the general solution. 


Solution: Earlier, we found the eigenvalues are 1,2,3. We found the eigenvector H for 


So 


the eigenvalue 3. Similarly we find the eigenvector Ea for the eigenvalue 1, and | for 


the eigenvalue 2 (exercise: check). Hence our general solution is 


1 0 1 ce’ + cge** 
#=c, |—-1| e+e. | 1 | ec +3 | 1] ce = | —cye’ + coe” + cye** 
0 = 0 tee" 


In terms of a fundamental matrix solution, 


et O. 6) fe 
r=X(i)je= |—e @ eo) |e 
0 =-e* O] Ie 


a 


Exercise 4.4.1: Check that this £ really solves the system. 


Overall, the process for finding the solution for real and distinct eigenvalues is to first 


find the eigenvalues and eigenvectors of the matrix P. Once we have these, we get n linearly 


independent solutions of the form z;(t) = v;e™', so that the general solution is of the form 


E(t) = c0e** + ete + ---+e,t,e. 


Then, if we need to solve for an initial condition, we figure out the coefficients ¢,, C2, ..., Cn 
to satisfy this condition. 

Note: If we write a single homogeneous linear constant coefficient n‘" order equation as a 
first order system (as we did in § 4.1), then the eigenvalue equation 


det(P — AI) =0 


is essentially the same as the characteristic equation we got in § 2.1 and § 2.7. See the 
exercises for details about this. 


Example 4.4.2: Solve the initial value problem 


if = ie ae 7(0) = He 
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Solution: Since we are in the case of a constant-coefficient linear system, we start by looking 
for the eigenvalues and eigenvectors of the coefficient matrix P. To do this, we compute 


det(P 21) = (0S 2)(7 =) = AS3) = 2 Ea 


This polynomial factors as (A + 3)(A + 4), and so the two eigenvalues are \; = —3 and 
Ag = —4. 

Next, we need to find the corresponding eigenvectors. For A = —3, we get the matrix 
equation 


(P +31e= e A a=6. 


The two equations that you get here are redundant, which is 3v,; + 4v9 = 0. One way to 
satisfy this is v1 = 4, ve = —3, so that the eigenvector is [ 43]. 


For \ = —4, the matrix becomes 
eit | Ao Age 
(P+4))¢v= eS {7-9 


so the eigenvector here is [ 1;]. Therefore, the general solution to this differential equation, 


by superposition, is 
2 2 aS tess 
Clty Sey | e + cp [7] er 


Finally, we have to solve the initial value problem using the initial conditions. If we plug 
in t = 0, we get the equation 


so -a[4]+a[4]-[]) 


This results in needing to solve the system of equations 
4¢q, +2 =1 — 3c) —@=l1. 


These can be solved in any way, including row reduction. We will start by adding the two 
equations together, which gives c; = 2, and then the first equation implies that co = —7. 
Therefore, the solution to the initial value problem is 


> _ 4 —3t 1 —4t __ 8e— 3 = Teo 
ae ei © |6e-3t + Te~4] ° 


4.4.2 Phase Portraits 


Now that we have these solutions, we want to get an idea for what they look like in the 
plane. We spent a lot of time in first order equations looking at direction fields, as well as 
phase lines for autonomous equations. We want to develope the same type of intuition for 
two-component systems in the plane, because much intuition can be obtained by studying this 
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simple case. Suppose we use coordinates (x, y) for the plane as usual, and suppose P = [25] 
is a 2 x 2 matrix. Consider the system 


orb] = Glee abl - 


The system is autonomous (compare this section to § 1.7) and so we can draw a vector field 
(see the end of § 4.1). We will be able to visually tell what the vector field looks like and how 
the solutions behave, once we find the eigenvalues and eigenvectors of the matrix P. The goal 
is to be able to sketch what the different trajectories of the solutions look like for a variety of 
initial conditions, as well as classify the general type of picture that results depending on the 
matrix P. 


Case 1. Suppose that the eigenvalues of P are real and positive. We find two corresponding 
eigenvectors and plot them in the plane. For example, take the matrix |} 4]. The eigenvalues 
are 1 and 2 and corresponding eigenvectors are [}] and [+]. See Figure 4.5. 

Suppose the point (x,y) is on the line 
determined by an eigenvector v for an eigen- 
value A. That is, [7] = adv for some scalar a. 
Then 


Al =P A = P(av’) = a(Pv) = aw. | Ca | 


The derivative is a multiple of v and hence 
points along the line determined by v. As 

A > 0, the derivative points in the direction 
of v when a is positive and in the opposite 
direction when a is negative. Let us draw the — =! + : , 4 , J 
lines determined by the eigenvectors, and let 

us draw arrows on the lines to indicate the 
directions. See Figure 4.6 on the following 
page. 

We fill in the rest of the arrows for the vector field and we also draw a few solutions. See 
Figure 4.7 on the next page. The picture looks like a source with arrows coming out from the 
origin. Hence we call this type of picture a source or sometimes an unstable node. Notice the 
two eigenvectors are drawn on the entire vector field figure with arrows, and the straight-line 
solutions follow them. 


3 -2 1 0 1 2 3 
3 3 


Figure 4.5: Eigenvectors of P. 


Case 2. Suppose both eigenvalues are negative. For example, take the negation of the 


matrix in case 1, [G! 23]. The eigenvalues are —1 and —2 and corresponding eigenvectors 


are the same, [4] and [+]. The calculation and the picture are almost the same. The only 
difference is that the eigenvalues are negative and hence all arrows are reversed. We get 
the picture in Figure 4.8 on the following page. We call this kind of picture a sink or a 
asymptotically stable node. 

Case 3. Suppose one eigenvalue is positive and one is negative. For example the matrix 


[6 2]. The eigenvalues are 1 and —2 and corresponding eigenvectors are [4] and [’;]. We 
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Figure 4.6: Eigenvectors of P with directions. Figure 4.7: Example source vector field with 
eigenvectors and solutions. 


ve ¢ 
be ¢ 
Vv ¢ 
7 a a 
vove 
vow + 
ee a 2 
Vovoyv 
any 

= a 


y 
V 
v 
v 
v 
Vv 
y 
v 
Ps 
z 
< 


A AIR TK £4 ZEU 


< = ‘<: < < = ra td > 
ee Se 1067 7 
A KR PF FP FF OF ¥ A 4 4 
a Lt ai on ee 1 e KAA 
A A KR N Bh OD x A AA 
A A A ON ON OD BS ho oN OA 
2 A A A HN AOD 2 X hh 
A AA Hh A ON \ yh 
fe. BP mt ®t & Fh 
4 PSA ee ce S ‘a ; 
3 2 1 0 1 2 3 3 -2 A 0 1 2 3 


Figure 4.8: Example sink vector field with Figure 4.9: Example saddle vector field with 
eigenvectors and solutions. eigenvectors and solutions. 


reverse the arrows on one line (corresponding to the negative eigenvalue) and we obtain the 
picture in Figure 4.9. We call this picture a saddle point. 


4.4.3. Exercises 


Exercise 4.4.2: 


a) Find the general solution of x, = 2x, x, = 3x2 using the eigenvalue method (first write 
the system in the form x’ = AZ). 


b) Solve the system by solving each equation separately and verify you get the same 
general solution. 
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Exercise 4.4.3: Find the general solution of x, = 321 + %2, v5 = 2x”, + 4x2 using the 
eigenvalue method. 


Exercise 4.4.4:* Solve x = x2, x = x using the eigenvalue method. 
Exercise 4.4.5: Consider the second order equation given by 
y” + 2y' — 8y = 0. 
a) Find the general solution of this problem using the methods of Chapter 2. 
b) Convert this equation into a first order linear system using the transformation ¢ = ra 


c) Find the eigenvalues and eigenvalues of the coefficient matrix and use that to find the 
general solution to the system. 


d) Extract the first component of the general solution and compare that to the solution 
from part (a). How do they relate? 


e) Look back through the work. How do the equation used to find the roots in (a) and 
the eigenvalues in (c) relate to each other? 


Exercise 4.4.6: Consider the second order equation given by 
y” +4y' + 5y = 0. 
a) Find the general solution of this problem using the methods of Chapter 2. 
b) Convert this equation into a first order linear system using the transformation t= ra 


c) Find the eigenvalues and eigenvalues of the coefficient matrix and use that to find the 
general solution to the system. 


d) Extract the first component of the general solution and compare that to the solution 
from part (a). How do they relate? 


e) Look back through the work. How do the equation used to find the roots in (a) and 
the eigenvalues in (c) relate to each other? 


Exercise 4.4.7: Consider the second order equation given by 
y + by’ — cy = 0. 
for b and c two real numbers. 
a) Find the general solution of this problem using the methods of Chapter 2. 
b) Convert this equation into a first order linear system using the transformation ¢ = Li 


c) Find the eigenvalues and eigenvalues of the coefficient matrix and use that to find the 
general solution to the system. 


d) Extract the first component of the general solution and compare that to the solution 
from part (a). How do they relate? 


e) Look back through the work. How do the equation used to find the roots in (a) and 
the eigenvalues in (c) relate to each other? 
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Exercise 4.4.8: Amino acid dating can be used by forensic scientists to determine the time 
of death in situations where other techniques might not work. These amino acids are sneaky, 
and they exist in a left-handed form (L) and a right-handed form (D), which are called 
enantiomers. While you're alive, your body keeps all your amino acids in the L form. Once 
you die, your body no longer regulates your amino acids, and every so often they flip a coin 
and decide whether to switch into the opposite form. This way, when someone finds your 
body in a dumpster, they can pull out your teeth and measure the racemization ratio, which 
is the ratio of D-enantiomers to L-enantiomers. 

Denote by D(t) and L(t), respectively, the proportions of D- and L-enantiomers found in 
your teeth, where t is measured in years after death. Since this is Math class, the proportions 
are governed by a system of differential equations, such as 


ve =z 02 L 
[|= [coe ne [5]. a 
a) Find the general solution to (4.4). 


b) Solve (4.4) with initial conditions D(0) = 0 and L(0) = 1, and express the solution in 
component form. Describe what happens to the quantities D(t) and L(t) in the long 
run. 


c) Given the above initial conditions, if the racemization ratio in your teeth is currently 
1:3, how long ago did you die? 


Exercise 4.4.9: 


9 2 6 
a) Compute eigenvalues and eigenvectors of A = E 3 5 | 


b) Find the general solution of x’ = Az. 
Exercise 4.4.10:* 


a) Compute eigenvalues and eigenvectors of A = E 6 i] . 
b) Solve the system #' = Az. 
abe 
Exercise 4.4.11: Let a,b,c,d,e, f be numbers. Find the eigenvalues of E d “|. 


Exercise 4.4.12:* Find the general solution of the system 


=, Ant —7 1 => 
BS | Oe 


Exercise 4.4.13:* Find the general solution of the system 


i. |e le 
~{9 8 |” 
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Exercise 4.4.14:* Find the general solution of the system 


=2.. -=6- 0 
Mea kB OH are 
—4 -7 3 


Exercise 4.4.15:* Find the general solution of the system 


-6 2 4 
f= |-2 -1 4} 72. 
=) |. 0 


Exercise 4.4.16: Solve the initial value problem 
v=|5 a2 z= [9 |. 


Exercise 4.4.17: Solve the initial value problem 


i= ; a £(0) = He 


Exercise 4.4.18: Solve the initial value problem 
7 4A O —3 


#@=|-8 -5 o|# z0)=| 2 
ian ae” 2 
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4.5 Eigenvalue method with complex eigenvalues 


Attribution: [JL], §3.4, 3.7. 


After this section, you will be able to: 


e Use Euler’s formula to find a real-valued general solution to a first order system 
with complex eigenvalues and 


e Solve initial value problems from all of these cases once the general solution has 
been found. 


As we have seen previously, a matrix may very well have complex eigenvalues even if 
all the entries are real. However, this may seem concerning going forward into solutions to 
differential equations that require these complex numbers in them. We will see in this section 
that we can still write solutions this way, but we no longer have straight-line solutions. Take, 


for example, 
ee ee Se 
t= [7 i}7 


Let us compute the eigenvalues of the matrix P = [ +, }]. 


1-2 1 


= pa)) =@-a8+1=8-242=0 


det(P — I) = det (| 


Thus \ = 1 +7. Corresponding eigenvectors are also complex. Start with A = 1 —7. 


The equations 7v,; + vo = 0 and —v, + iva = 0 are multiples of each other. This may be 

trickier to spot than the real version, but that is because they are complex multiples of each 

other. If we multiply the first equation by i, we get exactly the second one. So we only need 

to consider one of them. After picking v2 = 1, for example, we have an eigenvector @ = [{]. 

In similar fashion we find that [7'] is an eigenvector corresponding to the eigenvalue 1 + 7. 
We could write the solution as 


) — ) ) ql ijt —_ ) qd ijt 
s i i a F cyie Cole 
LS i el-9t + oy I | eer = : (1-)t is p(itiyt | ° 


We would then need to look for complex values c; and c2 to solve any initial conditions. It 
is perhaps not completely clear that we get a real solution. After solving for c; and cy, we 
could use Euler’s formula and do the whole song and dance we did before, but we will not. 
We will apply the formula in a smarter way first to find independent real solutions. 


In this case, we only needed one of the two eigenvectors to get the general solution, which 
happens because the complex eigenvalues and eigenvectors always come in conjugate pairs. 
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Z+Z 
2 


the bar above z means a + 7b = a—1b. This operation is called the complex conjugate. If a is 
a real number, then a = a. Similarly we bar whole vectors or matrices by taking the complex 
conjugate of every entry. Suppose a matrix P is real. Then P = P, and so P# = P# = Pz. 
Also the complex conjugate of 0 is still 0, therefore, 


First a small detour. The real part of a complex number z can be computed as , where 


0=0=(P_-ANe=(P-ADG. 


In other words, if \ = a + ib is an eigenvalue, then so is \=a—ib. And if vis an eigenvector 
corresponding to the eigenvalue A, then Vv is an eigenvector corresponding to the eigenvalue X. 
Suppose a+ ib is a complex eigenvalue of P, and v is a corresponding eigenvector. Then 


Ty _ elatibyt 


# and so 


is a solution (complex-valued) of x” = Pz. Euler’s formula shows that e#+# = e*— 
Lo = Z1 = ve 


is also a solution. As 7%, and £2 are solutions, the function 


+X, j%«14+7% 1 1 
Za, = ReZ, = Revelatt = = es z. 
£3 eX, eve 5 5 a + 52 


2-2 


is also a solution. And #3 is real-valued! Similarly as Im z = ** is the imaginary part, we 


find that 


> => 


TMT Ti 25 

BM 84 
is also a real-valued solution. It turns out that 73 and #4 are linearly independent. We will 
use Euler’s formula to separate out the real and imaginary part. 


r4 = Im 2} = 


Returning to our problem, 


. ; . . iB t . if . t 
z= H edit H cui an = i cost +e a _ |: at £4 i nn . 


1 e' cost — ie’ sint e' cost —e' sint 
Then 
Rez e'sint q Imz e' cost 
ez, = an mz) = ; 
e' cost] ’ =e sin? | * 


are the two real-valued linearly independent solutions we seek. 
Exercise 4.5.1: Check that these really are solutions. 


This gives that we can write the general solution to this problem as 


_ e'sint e' cost ce’ sint + coe’ cost 
w= Cy 2 tos = t tos . 
—e’sint cye’ cost — c9e" sint 


This solution is real-valued for real c; and cy. We now solve for any initial conditions we may 
have. Notice that the i has been dropped from the part of the process where we split the 
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complex solution into real and imaginary parts. The point is that the real and imaginary 
parts of the solution are independently solutions to the equation, and so we can use them to 
form our basis of solutions with constants c,; and c2 in front of them. We want everything to 
be real, and this process allows us to do it. 


Let us summarize as a theorem. 
Theorem 4.5.1 


Let P be a real-valued constant matrix. If P has a complex eigenvalue a + ib and 
a corresponding eigenvector v, then P also has a complex eigenvalue a — ib with a 


corresponding eigenvector 0. Furthermore, z = Pz has two linearly independent 
real-valued solutions 


and Zo = Im vest), 


The main point here is that the real and imaginary parts of these complex solutions are 
the real-valued independent solutions that we seek. Compare this to Theorem 2.2.2 in § 2.2, 
where we saw that the same idea worked for second order equation with complex roots. 

For each pair of complex eigenvalues a + 7b and a — ib, we get two real-valued linearly 
independent solutions. We then go on to the next eigenvalue, which is either a real eigenvalue 
or another complex eigenvalue pair. If we have n distinct eigenvalues (real or complex), then 
we end up with n linearly independent solutions. If we had only two equations (n = 2) as 
in the example above, then once we found two solutions we are finished, and our general 
solution is 

L= CL, + OF = C1 (Re delat*)*) + C2 (Im Berne), 


Example 4.5.1: Find the solution to the initial value problem 


7 = fee be # #0)= |.) | 


Solution: We start by looking for the eigenvalues and eigenvectors of the coefficient matrix. 
This results in the polynomial 


det(P — AI) = (1 —A)(—3 — A) — (4)(—2) = 17+ BA — A —3 +8 =A? + 2A +5. 
This polynomial does not factor, but the quadratic formula gives that the roots are 


peas ve OOO) = 1445" = 142% 


Thus, we are in the complex roots case, and can work from there. We need to find the 
complex eigenvector for one of these eigenvalues and then split into real and imaginary parts 
to get the general solution. 
For the eigenvalue A = —1 + 27, the matrix equation becomes 
~  |l—(-—1+ 23) 4 of. . [er 4 , > 
a =) ee ae 9! 9i|t=8 
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The two simultaneous equation that we need to solve for the vector v are 
(2 oo 21)v1 + Av, = 0 a 2U1 + (—2 = 21) v9 = 0 


and these equations don’t appear to be redundant. However, this is because they are complex 
multiples of each other, not just real multiples. To see this, we can multiply the first equation 
by the complex conjugate of the first coefficient. The idea is that if we do so, this first 
coefficient will be real, and then we can compare it to the second equation. If we multiply 
the first equation by 2 + 27, since (2 + 22)(2 — 27) = 8, it becomes 


8v1 + 4(2 + 21) v9 = 0 


and this is —4 times the second equation above. Therefore, they are redundant, and we can 
just pick one of them in order to find possible values of v; and v2. If we divide this newest 
equation by 8, it becomes 

vy + (1+ 4)v2 = 0. 


Based on this equation, we can pick vg = —1 and v; = 1+7. Therefore, the eigenvector for 
\ = —-1+4 2i is ['4']. This means that a complex-valued solution to this differential equation 
is 

ss Ra ae ee 

z(t) = | ee 


Now, we want to split this solution into real and imaginary parts in order to get a 
real-valued general solution. We apply Euler’s formula to do so: 


Z(t) = ' 4 | eaten 


= ' Es : e~'(cos(2t) + isin(2t)) 


ae ee +isin(2t) + icos(2t) — su 


— cos(2t) — isin(2t) 
- ia on Ba oe i | 


Therefore, we can take the real and imaginary parts of this solution to get a general solution 
as 


“ e~' cos(2t) — e~' sin(2t) e~‘ sin(2t) + e~' cos(2t) 
2 —e~' cos(2t) ug —e~' sin(2t) 
Exercise 4.5.2: Work out the eigenvector and general solution from eigenvalue \ = —1 — 22 


and verify that it is an equivalent general solution to the one above. 


Finally, we need to solve the initial value problem. Plugging in t = 0 gives 


so -a[!]+af]-[2] 
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The two equations that we get from here is cj + co = 1 and —c, = —2, so that c, = 2 and 
co = —1. Therefore, the solution to the initial value problem is 
ss e‘ cos(2t) — e* sin(2t)| _|e~* sin(2t) + e~* cos(2t) 2e~* cos(2t) — 3e~* sin(2t) 
rt) =2 _ pt _ _ ote = |_9,-+ -te 
e-‘ cos(2t) e-‘ sin(2t) 2e~* cos(2t) + e~* sin(2t) 


4.5.1 Phase Portraits 


Similarly to the real eigenvalue situation, we have three different cases for the phase portrait 
when the eigenvalues of a 2x2 matrix are complex. As metioned before, our basis solutions that 
we are using to form the general solution are no longer just exponential terms. They involve 
sines and cosines, and so are not straight lines anymore. Therefore, these solutions will not 
have straight lines in them, but we can still uses these basis solutions to help determine and 
describe the overall behavior of the solutions to the system for a variety of initial conditions. 

Case 1. Suppose the eigenvalues are purely imaginary. That is, suppose the eigenvalues 
are tib. For example, let P = [ ©, 4]. The eigenvalues turn out to be +2i and eigenvectors 
are [4,] and [_4,]. Consider the eigenvalue 27 and its eigenvector [4]. The real and imaginary 


parts of ve? are 


~(iJ°)- Lama) »(G~)- Le] 


We can take any linear combination of them to get other solutions, which one we take depends 
on the initial conditions. Now note that the real part is a parametric equation for an ellipse. 
Same with the imaginary part and in fact any linear combination of the two. This is what 
happens in general when the eigenvalues are purely imaginary. So when the eigenvalues are 
purely imaginary, we get ellipses for the solutions. This type of picture is sometimes called a 
center. See Figure 4.10. 
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Figure 4.10: Example center vector field. Figure 4.11: Example spiral source vector field. 
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Case 2. Now suppose the complex eigenvalues have a positive real part. That is, suppose 
the eigenvalues are a + ib for some a > 0. For example, let P = [ 1, {]. The eigenvalues turn 
out to be 1+ 2% and eigenvectors are [J] and [_5,;]. We take 1 + 27 and its eigenvector [ 4,] 
and find the real and imaginary parts of ve“+?" are 


Re(Jpife™™) =e sencan]> (ile) =¢ [pean | 


Note the e! in front of the solutions. The solutions grow in magnitude while spinning around 
the origin. Hence we get a spiral source. See Figure 4.11 on the preceding page. 


Case 3. Finally suppose the complex eigenvalues have a negative real part. That is, 
suppose the eigenvalues are —a + ib for some a > 0. For example, let P = [a rk The 
eigenvalues turn out to be —1 + 27 and eigenvectors are [_4;] and [4.]. We take —1 — 27 and 
its eigenvector [4,] and find the real and imaginary parts of ve~!~? are 


re([aferon) =e sencan]— P([a fe) = est) 


Note the e~ in front of the solutions. The solutions shrink in magnitude while spinning 
around the origin. Hence we get a spiral sink. See Figure 4.12. 
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Figure 4.12: Example spiral sink vector field. 


4.5.2 Exercises 


Exercise 4.5.3: Find the general solution of x, = 1, — 2%, vy = 2x1; + Xq using the 
eigenvalue method. Do not use complex exponentials in your solution. 


Exercise 4.5.4:* Solve x, = v2, v4 = —2, using the eigenvalue method. 


Exercise 4.5.5: A 2 x 2 matrix A has complex eigenvector U = H corresponding to 


eigenvalue A = —1 + 31. 
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a) Use Euler’s Formula to find the (real-valued) general solution to the system Z" = AZ. 
b) Sketch the phase portrait of this system. 
Exercise 4.5.6:* 
a) Compute eigenvalues and eigenvectors of A = { +, 9]. 
b) Solve the system #' = Az. 


Exercise 4.5.7: Consider the system 


a) Find the general solution. 


b) Solve the IVP with initial conditions x(0) = 1, y(0) = 0, and determine the maximum 
x-coordinate on this trajectory. 


Exercise 4.5.8: Find the general solution of the system 


, [4 1, 


Exercise 4.5.9: Find the general solution of the system 


grit Ale 


Exercise 4.5.10: Find the general solution of the system 


2 0 3 
f= |-6 2 —9| Zz. 
—-3 0 2 


Exercise 4.5.11: Find the general solution of the system 


—10 -4 0 
eli wa Ne 
2? 6 2 


Exercise 4.5.12: Solve the initial value problem 


# = i a e. ¢0)< el: 


Exercise 4.5.13: Solve the initial value problem 


zi = i ae #(0) = HE 


Exercise 4.5.14: Solve the initial value problem 
—-1 2 -8 2 
#=|/0 1 -4/2 TO) 
0 2 -83 —3 
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4.6 Eigenvalue method with repeated eigenvalues 


Attribution: [JL], §3.4, 3.7. 


After this section, you will be able to: 


e Find generalized eigenvectors to write a general solution to a first order system 
with repeated and defective eigenvalues, and 

e Solve initial value problems from all of these cases once the general solution has 
been found. 


There is one remaining case for the two-component first-order linear system: repeated 
eigenvalues. As we have seen previously, it may happen that a matrix A has some “repeated” 
eigenvalues. That is, the characteristic equation det(A — \J) = 0 may have repeated roots. 
This is actually unlikely to happen for a random matrix. If we take a small perturbation of 
A (we change the entries of A slightly), we get a matrix with distinct eigenvalues. As any 
system we want to solve in practice is an approximation to reality anyway, it is not absolutely 
indispensable to know how to solve these corner cases. On the other hand, these cases do 
come up in applications from time to time. Furthermore, if we have distinct but very close 
eigenvalues, the behavior is similar to that of repeated eigenvalues, and so understanding 
that case will give us insight into what is going on. 


Geometric multiplicity 


Take the diagonal matrix 
3 0 
fe FE | 

A has an eigenvalue 3 of multiplicity 2. We call the multiplicity of the eigenvalue in the 
characteristic equation the algebraic multiplicity. In this case, there also exist 2 linearly 
independent eigenvectors, [5] and [?] corresponding to the eigenvalue 3. This means that 
the so-called geometric multiplicity of this eigenvalue is also 2. These terms have all been 
discussed previously in § 3.6. 

In all the theorems where we required a matrix to have n distinct eigenvalues, we only 
really needed to have n linearly independent eigenvectors. For example, 7” = AZ has the 
general solution 


Let us restate the theorem about real eigenvalues. In the following theorem we will repeat 
eigenvalues according to (algebraic) multiplicity. So for the matrix A above, we would say 
that it has eigenvalues 3 and 3. 
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Theorem 4.6.1 


Suppose the n x n matrix P has n real eigenvalues (not necessarily distinct), A1, 2, 
.., An, and there are n linearly independent corresponding eigenvectors U1, U2, ..., Un- 
Then the general solution to 7’ = PZ can be written as 


Nae 


= = = wi + 
— cue! + cpvne?" + +++ + E_Une 


The main difference in the statement here from the theorem in § 4.4 is that we are no 
longer assuming that we have n distinct eigenvalues. Instead, we need to assume that we end 
up with n linearly independent eigenvectors, which we get for free if the eigenvalues are all 
distinct, but we might also have that if we do not have all distinct eigenvalues. 

The geometric multiplicity of an eigenvalue of algebraic multiplicity n is equal to the 
number of corresponding linearly independent eigenvectors. The geometric multiplicity is 
always less than or equal to the algebraic multiplicity. The theorem handles the case when 
these two multiplicities are equal for all eigenvalues. If for an eigenvalue the geometric 
multiplicity is equal to the algebraic multiplicity, then we say the eigenvalue is complete. 

In other words, the hypothesis of the theorem could be stated as saying that if all the 
eigenvalues of P are complete, then there are n linearly independent eigenvectors and thus 
we have the given general solution. 

If the geometric multiplicity of an eigenvalue is 2 or greater, then the set of linearly 
independent eigenvectors is not unique up to multiples as it was before. For example, for 
the diagonal matrix A = [23] we could also pick eigenvectors [+] and [',], or in fact any 
pair of two linearly independent vectors. The number of linearly independent eigenvectors 
corresponding to A is the number of free variables we obtain when solving Av = Av. We pick 
specific values for those free variables to obtain eigenvectors. If you pick different values, you 
may get different eigenvectors. 


Defective eigenvalues 


If an n x n matrix has less than n linearly independent eigenvectors, it is said to be deficient. 
Then there is at least one eigenvalue with an algebraic multiplicity that is higher than its 
geometric multiplicity. We call this eigenvalue defective and the difference between the two 
multiplicities we call the defect. 

a 1 

a 


has an eigenvalue 3 of algebraic multiplicity 2. Let us try to compute eigenvectors. 


b ol fal=9 


We must have that vg = 0. Hence any eigenvector is of the form [‘¢ |. Any two such vectors 
are linearly dependent, and hence the geometric multiplicity of the eigenvalue is 1. Therefore, 
the defect is 1, and we can no longer apply the eigenvalue method directly to a system of 
ODEs with such a coefficient matrix. 


Example 4.6.1: The matrix 
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Roughly, the key observation is that if \ is an eigenvalue of A of algebraic multiplicity m, 
then we can find certain m linearly independent vectors solving (A — AI )e = 0 for various 
powers k. We will call these generalized eigenvectors. 


Let us continue with the example A = [34] and the equation z’ = Az. We found an 
eigenvalue \ = 3 of (algebraic) multiplicity 2 and defect 1. We found one eigenvector U = [9]. 
We have one solution 

1 
#, = te* = | em 


0 


We are now stuck, we get no other solutions from standard eigenvectors. But we need two 
linearly independent solutions to find the general solution of the equation. 

Let us try (in the spirit of repeated roots of the characteristic equation for a single 
equation) another solution of the form 
Lo = vi te*, 


since our modified guess for repeated roots from second order equations was te®’. If we plug 
this guess into the equation, we get that 


zy = vie + 30,te™ 
and since the right-hand side of the equation is Av;te**, we need v, to satisfy 
vie" + 30, te** = Ai,te**. 


Since there is no e* term on the right-hand side of the equation, we are forced to pick % = 0, 
and so we get the solution 7 = 0, which is not good. This guess did not work. 

The issue here is that we didn’t have enough flexibility to actually get another solution to 
the differential equation, so we need something a little more complicated to make it work. 
To this end, we take a new guess of the form 


Eq = (ty + Ht) e**. 
We differentiate to get 
£, = Vre* + 3(v + Ut) e* = (30 + v1) e* + 3; te. 
As we are assuming that 7 is a solution, 7 must equal AZ. So let’s compute AZo: 
AZ, = A(¥q + vit) e* = Avge + Avite™. 


By looking at the coefficients of e* and te®! we see 302 + 0, = Avy and 30, = At;. This 
means that 
(A = 31) v2 = U1, and (A a 31)v; = 0. 


Therefore, 2 is a solution if these two equations are satisfied. The second equation is satisfied 
if | is an eigenvector, and we found the eigenvector above, so let % = [4]. So, if we can find 
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a U2 that solves (A — 3/)v2 = v), then we are done. This is just a bunch of linear equations 
to solve and we are by now very good at that. Let us solve (A — 3/)¥2 = v;. Write 


O 1) ja; {1 

0 O} |b} Jol” 
By inspection we see that letting a = 0 (a could be anything in fact) and b = 1 does the job. 
Hence we can take tj = [9]. Our general solution to 7” = AZ is 


3t 3t 
f= Cy A eet a C9 ((") he A ' ert -_ Re | 
2 


Let us check that we really do have the solution. First x) = c13e*! + cge*" + 3cote** = 321 +22. 
Good. Now x} = 3c2e** = 322. Good. 


In the example, if we plug (A — 3/)% = 0; into (A — 31), = 0 we find 
(A—3I\(A-31D%,=0, or (A-SI) =0. 


Furthermore, if (A — 31)w 4 0, then (A — 3/)w is an eigenvector, a multiple of @,. In this 
2x 2 case (A — 31)’ is just the zero matrix (exercise). So any vector w solves (A — 31)? = 0 
and we just need a w such that (A — 3)w@ 40. Then we could use w@ for %, and (A — 31) 
for Vj. 

Note that the system #’ = AZ has a simpler solution since A is a so-called upper triangular 
matriz, that is every entry below the diagonal is zero. In particular, the equation for x2 does 
not depend on x;. Mind you, not every defective matrix is triangular. 


Exercise 4.6.1: Solve i’ = [34] by first solving for x and then for x, independently. 
Check that you got the same solution as we did above. 


Let us describe the general algorithm. Suppose that A is an eigenvalue of multiplicity 2, 
defect 1. First find an eigenvector 0, of \. That is, o, solves (A — \J)v, = 0. Then, find a 
vector UV» such that 

(A — Al)v_ = Ui. 


This gives us two linearly independent solutions 


and so our general solution to the differential equation is 
Z(t) = cv,e™ + C2 (Uy + Vit) aa 


Example 4.6.2: Solve the initial value problem 


# = i | 2. a0) = ie 
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Solution: First, we need to look for the eigenvalues of the coefficient matrix. These are 


found by 
det(A = AT) S (—2 =a) (4 =A) — (3) (—8) SH 7 = 2A +150: 


Since this polynomial is (\ — 1)”, this has a double root at A = 1. 
For \ = 1, we can hunt for the eigenvector as solutions to 


—3 3 


aan [23 


Jeo 


These two equations are redundant, and the first equation is —3v; + 3v2 = 0, which can be 
solved by vy = vg = 1. Therefore, and an eigenvector for A = 1 is [+]. Thus, we have a 
solution to this system of the form 


Since we only found one eigenvector, we need to look for a generalized eigenvector as well. 
To do this, we want to solve the equation 


(A-NDw=¢ 
for the eigenvector U that we found previously. This means we need to solve 
—-3 3/ , {1 
Ps al? 
and both rows of the vector equation result in the equation —3w, + 3w. = 1 for w. We can 


pick any value of w, and wz to make this work. For the sake of this example, we will pick 
w, = 0 and w2 = 1/3. Then, we have that our second linearly independent solution to the 


differential equation is 
= 0 1 
#0 = ({i95) + [a] 9) ¢ 


and so the general solution to this system is 


z(t) = cy H eb + Cy ce + Fl ? e'. 


Finally, we can solve the initial value problem. Plugging in t = 0 gives 


Ay Ss, H icky, fe Z *] 


which gives that c, = 2 and then 2 + !/3co = —1, or cp = —9. Therefore, the solution to the 


initial value problem is 
aes. = ee 0 Ble \ ice 
Til S2 H e -9(],%5] + H tye 
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Exercise 4.6.2: We could have also chosen w; = —!/3 and w2 = 0 for the vector w. Use 
this to get a different looking general solution. Then solve the same initial value problem to 


see that you end up with the same answer at the end of the process. 
Example 4.6.3: Consider the system 
2 -5 0 
BS AO Be Oia. 
-1 4 1 


Find the general solution to this system using eigenvalues and eigenvectors. 


Solution: Even though this is a three-component system, the process is exactly the same: 
find the eigenvalues, compute corresponding eigenvectors, then build them together into a 
general solution. Compute the eigenvalues, 


2—-A -5 0 
0 = det(A — AL) = det 0 2-r OO = (2—)2(1—A). 
—l 4 1-2 
The eigenvalues are 1 and 2, where 2 has multiplicity 2. We leave it to the reader to find 
that H is an eigenvector for the eigenvalue A = 1. 
Let’s focus on A = 2. We compute eigenvectors: 
0 -5 0 V1 
0=(A-2I)v=|]0 0 O Ug 
—l 4 —l U3 
The first equation says that vg = 0, so the last equation is —v, — v3 = 0. Let v3 be the free 
variable to find that vj = —v3. Perhaps let v3 = —1 to find an eigenvector a |. Problem is 


that setting v3 to anything else just gets multiples of this vector and so we have a defect of 1. 
Let v, be the eigenvector and let’s look for a generalized eigenvector v9: 


(A = 21)v i U1, 


OF 25> SOP ake 1 
0: 0 oO Ot Sal gall. 
af A S2hil\\e = 


where we used a, b, c as components of V2 for simplicity. The first equation says —5b = 1 
so b = —1/5. The second equation says nothing. The last equation is —a + 4b —c = —1, or 
a+4/5+c=1, orat+c=1/s. We let c be the free variable and we choose c = 0. We find 
y 
U2 = =i) . 
0 
The general solution is therefore, 


0 1 1/5 1 
E=c; (Ole + oy) 0 ret bey | (ys) | 8) a) 
1 —1 0 
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ail 


This machinery can also be generalized to higher multiplicities and higher defects. We 
will not go over this method in detail, but let us just sketch the ideas. Suppose that A has 
an eigenvalue of multiplicity m. We find vectors such that 

(A—AD*e, =0, but (A-AD* ds, £0. 
Such vectors are called generalized eigenvectors (then v, = (A — AI)" ‘dp is an eigenvector). 
For the eigenvector v, there is a chain of generalized eigenvectors V2 through v; such that: 
(A — AI)v, = 0, 
(A = AT) v2 = V1, 


(A—AD)t, = Tet. 


Really once you find the #;, such that (A — AI)*%, = 0 but (A — AI)* '& 4 0, you find the 
entire chain since you can compute the rest, Up_1 = (A — Al) Ug, Ue_g = (A — AL) Ug-1, ete. 
We form the linearly independent solutions 

rt 


Z1 = vie 
= Lae ur 
t? tk-2 tk-l s 
Ey = | Up + Upit + Upon + °° +4 t Ui; 
k («, Uk-1 Uk 25 2h =O ia) ¢ 


Recall that k! = 1-2-3---(k —1)-k is the factorial. If you have an eigenvalue of geometric 
multiplicity @, you will have to find @ such chains (some of them might be short: just the 
single eigenvector equation). We go until we form m linearly independent solutions where m 
is the algebraic multiplicity. We don’t quite know which specific eigenvectors go with which 
chain, so start by finding w; first for the longest possible chain and go from there. 

For example, if \ is an eigenvalue of A of algebraic multiplicity 3 and defect 2, then solve 


(A-AD*®i=0, (A-ANH=%, (A-—ADiy = %. 


That is, find % such that (A — AJ)°03 = 0, but (A — AJ)*%3 4 0. Then you are done as 
Uo = (A — XAI)03 and Uv, = (A — AI)t. The 3 linearly eileen solutions are 


t? 
Zi = dye, Lo = (Uy + U;t) et £3 = (« + Vot + a5) et. 

If on the other hand A has an eigenvalue 4 of algebraic multiplicity 3 and defect 1, then 

solve 
(A —Al)v; = 0, (A — AI)v = 0, (A — AI)v3 = Vo. 

Here v, and V2 are actual honest eigenvectors, and v3 is a generalized eigenvector. So there 
are two chains. To solve, first find a 03 such that (A — \J)*03 = 0, but (A— AI)%; 4 0. Then 
U_ = (A—AI)¥3 is going to be an eigenvector. Then solve for an eigenvector Uv, that is linearly 
independent from v2. You get 3 linearly independent solutions 


Ss > Mt S = ~ dM 
Xi = Wye, £3 = (v3 + tot)e 


Ql 
SI 
o 
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4.6.1 Phase Portraits 


We also want to look at the phase portraits and direction field diagrams for repeated 
eigenvalues. There are two different options here, depending on if there are two linearly 
independent eigenvectors or not. 

Case 1. If we have a repeated eigenvalue with two linearly independent eigenvectors, this 
means that our matrix A is of the form 


vA 0 


Be NG. 


for the repeated eigenvalue A. This means that Av = Av for all vectors v. So, every vector is 
part of a straight line solution, and so every solution goes either directly towards or directly 
away from the origin. This gives a proper node which can be a sink or a source depending on 
whether the eigenvalue is positive or negative. 


Figure 4.18: Example proper nodal sink vector field. 


Case 2. If we have a repeated eigenvalue with only one linearly independent eigenvector, 
then we only have one straight-line solution. For instance, the matrix 


4 -1 


A=|1 9 


1 : : . 
1 for eigenvalue 3. Like the nodal sources and sinks, the 
solutions will go to zero and infinity along the straight line solutions. In this case, because 
there is only one straight line, the phase portrait looks somewhere between a node and a 
spiral. This gives an «improper node which can be a source or sink depending on the sign of 
the eigenvalue. 


has only one eigenvector of 


4.6.2 Exercises 


Exercise 4.6.3: Compute eigenvalues and eigenvectors of E 2 q : 
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i) 
mo 
xo 
N 
w 


Figure 4.14: Example improper nodal source vector field. 


Exercise 4.6.4: Let A = [3 ~j]. Find the general solution of z' = AZ. 


Exercise 4.6.5: Solve the initial value problem 


ope Bie ae TO 

z= ap #0) = 7). 
Exercise 4.6.6: Solve the initial value problem 

g-_ lo 22 ae Th 

e=[; 2 #0) = [3]. 


Exercise 4.6.7: Assume A is a3 x 3 matrix. The row-reduced echelon forms of A — XI are 
given for three different values of X: 


10 0 140 L = 1p 
A-3F~ {0 1 0 A-5I~ (001 A-7I~|0 0 0 
001 000 0 0 0 


Find the general solution of the homogeneous system Z' = AZ. 


7 5 -6 
Exercise 4.6.8: Consider the matrix A= |0 —3 2 
0-4 I 


a) Determine the characteristic polynomial of A and give its eigenvalues. 


b) How many (linearly independent) straight-line solutions does the system x" = AZ have? 
How do you know, without solving? 


1 5 -18 
Exercise 4.6.9: Let A= |2 —1 —5 
1 1 -6 
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1 
a) Show directly that v; = |3] is an eigenvector of A. 
1 


b) All eigenvalues of A are the same. Find the general solution to x’ = AZ. 


Exercise 4.6.10: Let A = 3, S 0 i; 


=a 
a) What are the eigenvalues? 
b) What is/are the defect(s) of the eigenvalue(s)? 


c) Find the general solution of z’ = AZ. 


1 

Li}, 

Ll 
a) What are the eigenvalues? 


b) What is/are the defect(s) of the eigenvalue(s)? 


Exercise 4.6.11:* Let A= | 


Ree 


c) Find the general solution of #' = AZ. 


Exercise 4.6.12: Let A = E 2 b). 


a) What are the eigenvalues? 
b) What is/are the defect(s) of the eigenvalue(s)? 


c) Find the general solution of x’ = Az in two different ways and verify you get the same 
answer. 


Exercise 4.6.13:* Let A= E i i 
a) What are the eigenvalues? 


b) What is/are the defect(s) of the eigenvalue(s)? 


c) Find the general solution of #' = Az. 


Exercise 4.6.14: Let A = E a) 2|. 


a) What are the eigenvalues? 
b) What is/are the defect(s) of the eigenvalue(s)? 


c) Find the general solution of z"’ = AZ. 


Exercise 4.6.15:* Let A= E “1 | : 


a) What are the eigenvalues? 
b) What is/are the defect(s) of the eigenvalue(s)? 


c) Find the general solution of #' = Az. 
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Exercise 4.6.16: Let A = F “4 a 


a) What are the eigenvalues? 
b) What is/are the defect(s) of the eigenvalue(s)? 


c) Find the general solution of z" = AZ. 
' 8 Choad 
Exercise 4.6.17: Let A= E 0.2 |. 
a) What are the eigenvalues? 


b) What is/are the defect(s) of the eigenvalue(s)? 


c) Find the general solution of <"’ = AZ. 
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Exercise 4.6.18: Suppose that A is a 2 x 2 matrix with a repeated eigenvalue \._ Suppose 


that there are two linearly independent eigenvectors. Show that A = AI. 


Exercise 4.6.19:* Let A = [¢¢], where a, b, and c are unknowns. Suppose that 5 is a 
doubled eigenvalue of defect 1, and suppose that [}| is a corresponding eigenvector. Find A 


and show that there is only one such matrix A. 


Exercise 4.6.20:* For each system, (i) classify the system according to type as one of 
sink/source/saddle/center/ spiral source/spiral sink; (ii) solve the systems; (iii) sketch the 


phase portrait. Both real and complex eigenvalues appear. 


Si 2 We Uo a 
je=(7 1)2 


b) a =( 
t= ( 
we=(S |, 
2 

( 


f) # = 
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4.7 ‘Two-dimensional systems and their vector fields 


Attribution: [JL], §3.65. 


After this section, you will be able to: 


e Visualize and sketch the behavior of a two dimensional system based on the 
eigenvalues and eigenvectors. 


In the last three sections, we looked at the different options for two-component constant- 
coefficient systems. We want to determine a nice way to put all of this together. We summarize 
the behavior of linear homogeneous two-dimensional systems given by a nonsingular matrix 
in Table 4.1. Systems where one of the eigenvalues is zero (the matrix is singular) come up 
in practice from time to time, see Example 4.1.2 on page 271, and the pictures are somewhat 
different (simpler in a way). See the exercises. 


Eigenvalues Behavior 

real and both positive source / unstable node 

real and both negative sink / asymptotically stable node 
real and opposite signs saddle 

purely imaginary center point / ellipses 


complex with positive real part spiral source 

complex with negative real part spiral sink 

repeated with two eigenvectors proper node (asympt. stable or unstable) 
repeated with one eigenvector improper node (asympt. stable or unstable) 


Table 4.1: Summary of behavior of linear homogeneous two-dimensional systems. 


The sketches of all of these different behaviors and phase portraits can be found in their 
respective sections. Make sure that you understand the terminology, general behavior, and 
sketches for each of these different cases. 


4.7.1 Trace-Determinant Analysis 


One other way to interpret and analyze this information is using the trace and determinant 
of the matrix. Recall from § 3.7 that the trace of a matrix is the sum of the diagonal entries 
of the matrix and the determinant of the matrix is computed from the entries and is a way 
to determine invertibility of the matrix. If we take a generic 2 x 2 matrix and find the 
characteristic polynomial, we get that for 


a b 
a=[0 a 
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the characteristic polynomial is 
det(A — AI) = (a — A)(d — A) — (8) (ec) = 7 — (a + d)A}+ (ad — be). 


Since the trace of the matrix is a+ d and the determinant is ad — bc, we can rewrite this 
polynomial as 
v= Ths D =, 


which also means that we can characterize the eigenvalues of the matrix in terms of the trace 
and determinant. We get that the eigenvalues are 


_T+VT?—4D 
= ; 


There are a few important facts we can learn from this equation. 


A 


(4.5) 


1. A lot depends on the value of T? — 4D. If T? — 4D > 0, then we will have two real 
distinct eigenvalues. If T? — 4D = 0, then there is a single repeated eigenvalue, and if 
T? —4D < 0, we have complex eigenvalues. 


2. If D <0, then T? — 4D > T?, which means that /T? — 4D > |T|. If we put this into 
(4.5), this will mean that the term that is after the + will be larger than T in absolute 
value. Therefore, the two eigenvalues will be real and have opposite signs. 


3. If D > 0, then the sign of the eigenvalues, or the sign of the real part in the complex 
case, is dictated by the sign of T. If D > 0, then T? — 4D < T?, so that the part under 
the square root in (4.5) is always smaller in absolute value than 7. Thus, both the plus 
and minus version will have values that are the same sign as 7’. If the expression is 
complex, then the real part is exactly T'/2, which is the same sign as T. 


All of this means we can make a new table characterizing the eigenvalues and how they 
are connected to the trace and determinant. 


Eigenvalues Trace and Determinant Classification 
real and both positive T>0,D>0,T?-4D>0 

real and both negative Pat, DS 17? =4)s0 

real and opposite signs D<0 

purely imaginary T=, D>0 


complex with positive real part JT >0,T?—4D <0 
complex with negative real part T <0, T?—4D <0 
repeated T?=AD=0 


Table 4.2: Summary of behavior of linear homogeneous two-dimensional systems. 


Since these are all based on the relation between 7 and D, we can also combine all of 
this into a figure to summarize the details. In Figure 4.15 on the following page, T’ is on 
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Spiral Sink Spiral Source 


Improper Improper 

Nodal Sink Nodal Source 
‘ ; 

Nodal Sink NO | Nodal Source * 


oe 


Saddle Saddle 


Figure 4.15: Trace-Determinant plane for analysis of two-component linear systems. 


the horizontal axis and D is the vertical axis. The graph drawn is D = T’/4, which is the 
important criteria that shows up in the table. 

Figure 4.15 can be used to determine the behavior of a two-component system without 
actually needing to solve the differential equation. The point is that the signs and type of 
the eigenvalues determine the structure of the solution, and we can determine the important 
qualities of these using just the trace and determinant of a matrix. 


Example 4.7.1: Use Trace-Determinant analysis to determine the overall behavior of the 
system 
1 4 
pe 2 
= 9 | ve 


Solution: From the matrix, we can see that the trace is 1 + 3 = 4 and the determinant is 
(1)(3) — (4)(—2) = 11. We see that D > 0 with T? = 16 and 4D = 44 > 16. Therefore, we 
have 4D > T?, so we are above the curve on the graph, and so have a spiral. Since T > 0, 
this will be a spiral source. 

Note: If you wanted to get a general solution or sketch a phase portrait for this differential 
equation, you would need to actually solve it out for that; you can not get enough information 
just from this image to sketch a proper phase portrait. | 
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Exercise 4.7.1: Compute the eigenvalues for the system above, find the general solution, 
and verify that this is a spiral source. The numbers here will not work out great, so having 
the quick analysis that it is a spiral source is nice. 


4.7.2 Exercises 


Exercise 4.7.2: Take the equation mx” + ca’ + kx = 0, with m >0,c>0,k > 0 for the 
mass-spring system. 


a) Convert this to a system of first order equations. 
b) Classify for what m,c,k do you get which behavior. 


c) Can you explain from physical intuition why you do not get all the different kinds of 
behavior here? 


Exercise 4.7.3: What happens in the case when P = |} +]? In this case the eigenvalue is 
repeated and there is only one independent eigenvector. What picture does this look like? 


Exercise 4.7.4: What happens in the case when P = [+ }]? Does this look like any of the 
pictures we have drawn? 


Exercise 4.7.5:* Describe the behavior of the following systems without solving: 


ajal’=at+y, y=r-y. b) a, =2it22, £5 = 229. 


/ — 


a ee ei d) a’ =2+3y, yf =—2a—Ay. 


/ 


e) a’ =x—4y, y =—4r+y. 


Exercise 4.7.6: Which behaviors are possible if P is diagonal, that is P = [#9]? You can 
assume that a and b are not zero. 


Exercise 4.7.7:* Suppose that #' = AZ where A is a 2 by 2 matrix with eigenvalues 2 + 7. 
Describe the behavior. 


Exercise 4.7.8:* For each of the following matrices A, describe the behavior and classify 
the phase portrait of the system given by x’ = Ax. Use the eigenvalues to determine this. 


a) A= i b) A= i i 
) A= [ao] ya=|y W5 
yA=[) o5] pa=[P 


Exercise 4.7.9: For each of the matrices and systems in Exercise 4.7.8, perform the same 
analysis using the trace and determinant of the matrix. 
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Exercise 4.7.10: Consider the system of differential equations given by 
St —2 —3 > 
=|, _o|/f 
a) Use trace and determinant analysis to determine the behavior of this linear system. 


b) Find the general solution to this system of differential equations and verify that it 
matches the analysis in (a). 


Exercise 4.7.11: Consider the system of differential equations given by 
ve —2 -3] 
=|5 4 |% 


a) Use trace and determinant analysis to determine the behavior of this linear system. 


b) Find the general solution to this system of differential equations and verify that it 
matches the analysis in (a). 
Exercise 4.7.12: Take the system from Example 4.1.2 on page 271, x, = (x2 — 21), 
@y = 7(x1 — £2). As we said, one of the eigenvalues is zero. What is the other eigenvalue, 
how does the picture look like and what happens when t goes to infinity. 


Exercise 4.7.13:* Take [7 ]’ = [°4][{]. Draw the vector field and describe the behavior. 
Is it one of the behaviors that we have seen before? 


Exercise 4.7.14: In this exercise, we will analyze “perturbations” or near-by matrices to 
the ones that are given. This will be important later in § 5.1. For each of the following 
matrices 


a) Find the trace and determinant, and use them to classify the behavior of the linear 
system x = Az for the given matrix A. 


b) Draw a sketch of the trace-determinant plane, including the curve D = T*/4, and plot 
the point corresponding to the matrix on those axes. 


c) Look at the points in a small (as small as you want) circle around the point you just 
drew. What does the behavior look like for systems whose matrices fall within that 
circle? What do these behaviors have in common with each other, and how do they 
differ? 
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4.8 Nonhomogeneous systems 


Attribution: [JL], §3.9. 


After this section, you will be able to: 


e Use the eigenvector decomposition or diagonalization to solve non-homogeneous 
systems, 
e Use undetermined coefficients to solve non-homogeneous systems, and 


e Use variation of parameters and fundamental matrices of solutions to solve 
non-homogeneous systems. 


Now, we want to take a look at solving non-homogeneous linear systems. As discussed 
previously, the process here is the same as it was for second order non-homogeneous equations. 
We can solve the homogeneous equation and then need one particular solution to the 
non-homogeneous problem. Adding these together gives the general solution to the non- 
homogeneous problem, where we can pick constants to meet an initial condition if it is given. 
This section here will focus on a variety of methods to find this particular solution. 


4.8.1 First order constant coefficient 


Diagonalization 


Diagonalization is a linear algebra-based process for adjusting a matrix into one that is 
diagonal. In order to see why this might be helpful in the process of solving non-homogeneous 
systems, or generating a particular solution to the non-homogeneous system, let’s start by 
looking at a problem with a diagonal matrix to see how we could solve it. 


Example 4.8.1: Find the general solution of the non-homogeneous system 
se — bd Oleg ee 
z= |) 3) 2+ | ot]: 
Solution: If we write this system out in components, we get 
A is Et e" 
= + 
5 0 3] |xe mal 


vo=a2t+e* 2 =3r.+e%. 


or 


These are two completely separated, or decoupled equations. We can solve each of these 
via first-order integrating factor methods. For the first, we get 
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and for the second, we see that 


1 
eo as = oo + Cy 
1 
Lo(t) = ae + Ce 


or, rewriting in a different form, 


eke yee Lil 3 OF) 433 
tt) = | +C; A e+C, H e". 


al 


Therefore, if we have a non-homogeneous system with a diagonal matrix, then we can 
separate the decoupled equations, solve them individually, and put them back together into 


a full solution. In this particular case, the eigenvectors of A were A and H , and so the 


standard basis vectors were the directions in which A acts like a scalar. When the eigenvectors 
are not the standard basis vectors, we need to take them into account in order to use this 
method. 

Take the equation 


z(t) = Az(t) + f(t). (4.6) 
Assume A has n linearly independent eigenvectors VU}, V2,..., Un, with corresponding eigenvalues 
Ay, A2,--+;An- Build the matrices 
de i> Ses 
Pe 7 Oy Agee 20 
E = [v; | |---| tn] DEW ie de ae ca A 
O: - Ae he 


that is, EF is the matrix with the eigenvectors as columns, and D is a diagonal matrix with 
the eigenvalues on the diagonal in the same order as the eigenvectors are put into F. Since 
we have n eigenvectors, both of these are n x n square matrices. It is a fact from linear 
algebra that 

A=EDE" or D=E'AE. 


a=[e 


compute the matrices E and D and verify that EDE~! = A. 


Exercise 4.8.1: For the matrix 
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With this tool in hand, we look to approach our non-homogeneous system. We would 
like for the system to use the matrix D instead of the matrix A, because that is decoupled 
and we can solve it directly. To do this, we define a new unknown function y by the relation 
<= Ey. If we plug this into (4.6), we get 


Eg (t) = AEG(t) + fle). 
Using the relation for A and the fact that E is a constant matrix, we get that 
By (t) = EDE“Bg(t) + Flt) = EDG(t) + Fl. 
If we multiply both sides of this equation by E~', we get 
7 (t) = Dyt) + Eft) 


and this is now a decoupled system of equations. Once we compute £7! fi (t), we can then 
solve this directly because it is based on a decoupled system of differential equations to solve 
for the solution y. Once we have 7, we can compute % as X = Ey to get our solution. 


Example 4.8.2: Let A =[}3]. Solve # = A% + f where f(t) = [26°] for #(0) = Eval 


Solution: The first step in this process is always to find the eigenvalues and eigenvectors of 
the coefficient matrix. We do this in the standard way 


det(A — AF) = (1 — A)(1 — A) — (3)(3) = 27 — 244-1-9 =)? - 20-8. 


Since this factors as (A + 2)(A — 4), the eigenvalues are —2 and 4. Using these (exercise!) 
we can show that the corresponding eigenvectors are | ',|] for X = —2 and [}] for \ = 4. 
Therefore, the general solution to the homogeneous problem is 


1 1 
Ge) Sey ] et Gs H e*. 


Now that we have this solution, we can work to solve the non-homogeneous problem. To 
do this, we form the matrices 


Li] [3 


and, using the fact that for a 2 x 2 matrix 


ab) 1 fd -b 
ec d| ad—bc|—-c a 


we can compute E~! as 


P= aa-menh ihe 7) 
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As an aside, we can check that A = EDE™! to make sure that we did this right. 


EDE |= ; i 


Ke 
Ke 


pe leary 


Thus, we can proceed. From the general process of diagonalization, we know that the 
system we need to solve is 


fn sorte. TP Nia. yes SSO oe! 
—f lf _ 
ee aie F=[ al @+ [i/o ales 
for y¥ = E~'2Z, or ¥ defined by % = Ey. Computing the non-homogenoeus term gives 
1/2 —1/2] |2e’| — je’-¢ 
1/2 1/2] |2t) jet+t 
so that we can now decouple the system 
vi =2. Ola St 
P| > co t 
into two separate first-order equations that we can solve 


y, =—2yite’—-t Yo =4yote'+t 
by normal first-order integrating factor methods. For the y, equation, we want to use an 
integrating factor of e?* to solve it as 
y, +2y,=e'-t 
erty! + 2e%y, = e3 — te 


(cy) =e — te 
1 1 1 
2t 3t 2t 3t 2t 2t 
e€ = | e* —te” dt= — -te” 4 + C 
M1 i. ger neh Ge 1 
Tee ie 
Se St aCe 
Y1 3 5) zi 1€ 
For the second, we need the integrating factor e~* to solve 
yy —4y2=e' +t 
ety! — de "yn = e734 te 
ety = pe Ate dt = _1,-3¢ = lett - Ae +C, 
3 4 16 
1 1 1 
a ee eee een OPT: ad 
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Therefore, we have the vector solution 


1.t 1 1 —2t 
e’ — 5t- + Ce 

u(t = 3 2 4 

y(t) Me tiy td See 


To get to the actual solution 7, we need to multiply this solution by the matrix E 
Ly Sf tees Se eC ee 
roar [ Yd 
—1 1 6 T 
a ge’ — pt + 44+ Cie* + (-Fe! 
ea 


_ —3t + 2 + Ce + Cae * 
~ |—2e! — at = % = Ge? + Cae 
which is a valid way to write the general solution. We can also write this solution in the form 
= OS lstty i a ee 
708 Ne g}e+ t t+ 16 +C; e + Cy € 
oa oa —i6 —l 1 
and we see that the general solution to the homogeneous problem shows up at the end of this 


solution. 
Finally, we need to satisfy the initial conditions. If we plug in t = 0, we get 


(0) = |. +0+ 8] Oy | + Cy H ~ EA 


Rearranging this expression gives the two equations 


2 
C,+C, =0 CTO Ss 
which has solution C, = —!/3 and Cy = 1/3. Therefore, the solution to the initial value 


problem is 
_3 3 1 1 
ct) = | e+ i]t “| —= | “feet g Hl en, 
“3 as —is| 3 t-t 3 [I | 


Another way to view this process is by thinking about it as eigenvector decomposition. 
(This approach is not necessary on a first reading. The next new information starts at 
the undetermined coefficients section.) The eigenvectors of A are the directions in which 
the matrix A basically acts like a scalar. If we can solve the differential equation in those 
directions, then it acts like a scalar equation, which we know how to solve. We can then 
reorient everything to get back to our original solution. 

Again, we start with the equation 


i (t) = Az(t) + f(t) (4.7) 
and assume A has n linearly independent eigenvectors U1, V2,..., Un, Write 


E(t) = 0 E(t) + Uo Eo(t) +--+ tr En(t). (4.8) 
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That is, we wish to write our solution as a linear combination of eigenvectors of A. If we 
solve for the scalar functions €, through €,, we have our solution 7. Let us decompose f in 
terms of the eigenvectors as well. We wish to write 


f(t) = U1 g(t) + Uo go(t) ++++ + Un Gn(t). (4.9) 


That is, we wish to find g, through g, that satisfy (4.9). Since all the eigenvectors are 
independent, the matrix FE =[v, v2, --- J, | is invertible. Write the equation (4.9) as 
‘a = Eg, where the components of g are the functions g; through g,. Then g = E7! Z. Hence 
it is always possible to find g when there are n linearly independent eigenvectors. 

We plug (4.8) into (4.7), and note that Av, = A,Uz: 


_ 


a At f 
iE) + Valo +--+ + OnE, = A (Tir + Gabe +--+ nbn) + igi + Gage +--+ tngn 
= Aviés + Aveo +++ + AtnEn + Uigi + Voge + +++ + Ungn 
= UyA11 + VoA2bs + +++ + GnAn&n + Vigi + Vage +--+ + GnGn 
= U1(A1€1 + 91) + Va(Avk2 + ge) +++ + Un(AnEn + gn): 


If we identify the coefficients of the vectors v; through v,, we get the equations 


& =A18i+ 91, 
& = A2&o + 92, 


Each one of these equations is independent of the others. They are all linear first order 
equations and can easily be solved by the standard integrating factor method for single 
equations. That is, for the k'* equation we write 


E,(t) — An€e(t) = ge(t). 
We use the integrating factor e~**' to find that 
d 
zr Ext) eel =e *g, (t). 


We integrate and solve for &; to get 
&,(t) = er** / eta, (£) dt + Cye**. 


If we are looking for just any particular solution, we can set Ci, to be zero. If we leave these 
constants in, we get the general solution. Write 7(t) = 1&1 (t) + Go€o(t) +--- + 0nE,(t), and 
we are done. 
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As always, it is perhaps better to write these integrals as definite integrals. Suppose that 
we have an initial condition 7(0) = b. Take @ = E~'b to find b = Ua, + Boag + +++ + Tran, 
just like before. Then if we write 


t 
| Ex(t) = an) e***gx(s) ds + aye, 
0 


we get the particular solution #(t) = o,€(t) + €2(t) +---+ TE,(t) satisfying Z(0) = 6, 
because &;(0) = ag. 

Let us remark that the technique we just outlined is the eigenvalue method applied to 
nonhomogeneous systems. If a system is homogeneous, that is, if 7 = 0, then the equations 
we get are € = A,€z, and so & = C,e**' are the solutions and that’s precisely what we got 
in § 4.4. 


Example 4.8.3: (Same as the previous example) Let A = [}3]. Solve z = Az + f where 
F(t) = [3] for 2(0) = [ 47, ]. 
Solution: The eigenvalues of A are —2 and 4 and corresponding eigenvectors are [_',] and 


[+] respectively. We write down the matrix FE of the eigenvectors and compute its inverse 
(using the inverse formula for 2 x 2 matrices) 


ft, 4 4 Uf 
e= |", a ie =5\: ae 


We are looking for a solution of the form # = [+,]; +[+]&. We first need to write ea 
z | a [j}a + [+] go. Thus 


> 


terms of the eigenvectors. That is we wish to write f = 


So 9g, =e’ —t and g =e’ +t. 
We further need to write Z(0) in terms of the eigenvectors. That is, we wish to write 
#(0) = | 278.) = [Jai + [Ha Hence 


am} | fe} _ | 1/4 
eee Fied= Pi 


So a, = !/4 and ag = —!/16. We plug our Z into the equation and get 


z’ Ag f 


1 1 1 1 1 
fe e+ H =A ] f+A H fo + ] Cia H 92 
1 


2 ] (—261) + H dE + ] (e’-t)+ H (et +1). 
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We get the two equations 


1 
f= Dee where £:(0) = a1 = 7, 

=I 
€h=Aftet+t, where 2(0) = a2 = =. 


We solve with integrating factor. Computation of the integral is left as an exercise to the 
student. You will need integration by parts. 
5e 2t (ot og | Se a 
€) =e e“(e —t)dt+Cie ~ = ——--~+-+4+Cie™. 
a. Qa 
C; is the constant of integration. As (0) = 1/4, then 1/4 = 1/3+ 1/4+ Cy and hence Cy = —1/s. 
Similarly 
fo =e |e (eb +t) dt + Coe® = ota s 
‘ : a 4° 6 


As &9(0) = —1/16 we have —1/16 = —1/3 — 1/16 + Cp and hence Cy = 1/3. The solution is 


t —2t At t ett_e—2t 3-12 
z(t) = 1 e’—e ib 2b Fe Ser Abad 7 a se 
=] 5 I 4 ! 1 3 16 e~2t+e4t_ et 4t—5 
a a, 


3 16 


1 2 
é At_,—2t BY —2t ,4t_9,t _ 
That is, 2) = > + 3 aa and rz = £ +6 Lae a. | 


Exercise 4.8.2: Check that x, and x2 solve the problem. Check both that they satisfy the 
differential equation and that they satisfy the initial conditions. 


Undetermined coefficients 


The method of undetermined coefficients also works for systems. The only difference is that 
we use unknown vectors rather than just numbers. Same caveats apply to undetermined 
coefficients for systems as for single equations. This method does not always work for the same 
reasons that the corresponding method did not work for second order equations. We need to 
have a right-hand side of a proper form so that we can “guess” a solution of the correct form 
for the non-homogeneous solution. Furthermore, if the right-hand side is complicated, we 
have to solve for lots of variables. Each element of an unknown vector is an unknown number. 
In system of 3 equations with say say 4 unknown vectors (this would not be uncommon), we 
already have 12 unknown numbers to solve for. The method can turn into a lot of tedious 
work if done by hand. As the method is essentially the same as for single equations, let us 
just do an example. 


=> 


Example 4.8.4: Let A = [739]. Find a particular solution of # = A+ f where f(t) = 
“1. 

Solution: Note that we can solve this system in an easier way (can you see how?), but for 
the purposes of the example, let us use the eigenvalue method plus undetermined coefficients. 
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The eigenvalues of A are —1 and 1 and corresponding eigenvectors are [+] and [9] respectively. 
Hence our complementary solution is 


fmalieralile 
o=Cy 1 e +0€2 l e, 


for some arbitrary constants c, and C9. 
We would want to guess a particular solution of 


@= de! +bt+d. 

However, something of the form de’ appears in the complementary solution. Because we do 
not yet know if the vector @ is a multiple of [9], we do not know if a conflict arises. It is 
possible that there is no conflict, but to be safe we should also try kte’. Here we find the 
crux of the difference between a single equation and systems. We try both terms de’ and kte! 
in the solution, not just the term kte’. Therefore, we try 

& = Ge'+kte’+bt+d. 
Thus we have 8 unknowns. We write @ = bal b= EE k= Ee and d= EE We plug 


# into the equation. First let us compute 2”. 


= (a k) e+ kte’+b= *: is “| ea fn te’ + Be 


a2 ko ko 
Now 2 must equal Az’ + f, which is 


Az + f = Ade’ + Aktet + Abt + Ad + f 
- —ay, t —ky t —b, —d, UW) gee EO) 
ee ae | oer es ls dl tery es aie 4 ae Loe + 4 2 i ae H : 
_ —a, 7 1 t —ky, t —b; —d, 
- eae +r a oe ae +r | ls Ee a by + A Ger lea + dy ; 


We identify the coefficients of e’, te’, t and any constant vectors in @ and in Ag + f. to find 
the equations: 


a, +k) = —-a, +1, 0 = —by, 

dg + kp = —2a, + ag, 0 = —2b, +b. +1, 
ky =—k, b, = —di, 
kg = —2k, + ko, by = —2d, + do. 


We could write the 8 x 9 augmented matrix and start row reduction, but it is easier to just 
solve the equations in an ad hoc manner. Immediately we see that k, = 0, b; = 0, d; = 0. 
Plugging these back in, we get that bg = —1 and dy = —1. The remaining equations that tell 
us something are 


ay = —ay, + 1, 
ag + ko = —2a, + a2. 
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So a, = !/2 and kg = —1. Finally, ag can be arbitrary and still satisfy the equations. We are 
looking for just a single solution so presumably the simplest one is when az = 0. Therefore, 


at oe te ee ee t 0 Ole: ae 
Z = de’ + kie +it+d= |i jer[E nt |S] e+ |S] = vet ‘ 


That is, 7; = se, 2 = —te' —t — 1. We would add this to the complementary solution to 
get the general solution of the problem. Notice that both de’ and kte' were really needed. | 


Exercise 4.8.3: Check that x; and x2 solve the problem. Try setting ag = 1 and check we 
get a solution as well. What is the difference between the two solutions we obtained (one 
with az = 0 and one with ay = 1)? 

As you can see, other than the handling of conflicts, undetermined coefficients works 
exactly the same as it did for single equations. However, the computations can get out of 
hand pretty quickly for systems. The equation we considered was pretty simple. 


4.8.2 First order variable coefficient 


Variation of parameters 


Just as for a single equation, there is the method of variation of parameters. This method 
works for any linear system, even if it is not constant coefficient, provided we somehow solve 
the associated homogeneous problem. 

Suppose we have the equation 


# = A(t) #+ fit). (4.10) 


Further, suppose we solved the associated homogeneous equation z’ = A(t) # and found a 
fundamental matrix solution X(t). If we find separate, linearly independent solutions, this 
matrix X(t) can be generated by putting these solutions as the columns of a matrix. The 
general solution to the associated homogeneous equation is X (t)¢ for a constant vector ¢. Just 
like for variation of parameters for single equation we try the solution to the nonhomogeneous 
equation of the form 

Z, = X(t) a(t), 


where u(t) is a vector-valued function instead of a constant. We substitute z, into (4.10) to 


obtain - 
X'(t) u(t) + X(t) w(t) = A(t) X(t) a(t) +f (0). 
————————_ —_—____ 

#,(t) A(t)€p(t) 


But X(t) is a fundamental matrix solution to the homogeneous problem. So X’(t) = A(t) X(t), 


and 

XGA + X(t) w(t) = X1B AH + F(t) 
Hence X(t) w(t) = f(t). If we compute [X(t)]’, then a(t) = [X(t)] | f(t). We integrate to 
obtain w and we have the particular solution 7, = X(t) u(t). Let us write this as a formula 
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Example 4.8.5: Find a particular solution to 


1 t -l t 
> > 2 
f= Pal i : | E+ H (t“ + 1), (4.11) 


given that the general solution to the homogeneous problem 


to a t —1 > 
aoe SG ee ale 


Pas, H ee Ei . 


is 


Solution: Here A = aa [; ;'] is most definitely not constant, so it’s a good thing that we 
have the general solution to this system. From this, we can build the matrix X(t) as 
L =t 
aed 


, which is a fundamental matrix for this system and solves X'(t) = A(t)X(t). Once we know 
the complementary solution we can find a solution to (4.11). First we find 


Ol =asz|2) ap 


Next we know a particular solution to (4.11) is 


Adding the complementary solution we find the general solution to (4.11): 
= 1 -t Cy 7 a Cy — Cot + zt 
a -+ 2 43 = 243|° 
t 1 [ee gee er lar liars t 


Exercise 4.8.4: Check that x, = §t* and x2 = 3t? +t really solve (4.11). 
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In the variation of parameters, we can obtain the general solution by adding in constants 
of integration. That is, we will add X(t)¢ for a vector of arbitrary constants. But that is 
precisely the complementary solution. 

To conclude this section, we will solve one example using all three methods to be able to 
compare and contrast them. All of them have their benefits and drawbacks, and it’s good to 
be able to do all three to be able to choose which to apply in a given circumstance. 


Example 4.8.6: Find the general solution to the system of differential equations 
4  |-5 —2 #4 ery 
seal as ea ced 

Solution: No matter which of the three methods we want to use to solve this problem, we 


always need the eigenvalues and eigenvectors of the coefficient matrix in order to find the 
general solution to the homogeneous problem. These are found by 


det(A — AT) = (—5 — A) = A) = (—2)(4) =? +44 —548 =? 4043. 


This polynomial factors as (A+ 1)(A +3) so the eigenvalues are —1 and —3. For A = —1, the 
system we need to solve is 


~ |-4 -2/_ = 
(a+De= | » | e=0 
which can be solved by the vector v = [ 1,]. For \ = —3, the system is 
~ |-2 -2/_ = 
arane=[ j|e=0 
which can be solved by the vector v = [ ",]. Therefore, the general solution to the homogeneous 
problem is 
z(t) =C, |.) PE ee 1 ee, (4.12) 


Now, we can divide into the different methods that we want to use to solve the non- 
homogeneous problem. 


1. Diagonalization. For this method, we need the matrices EF and D defined by 
1 1 —1 O 
Ba Plo 


and can then compute E~! as 


We then compute 


4.8. NONHOMOGENEOUS SYSTEMS 341 


which gives rise to the decoupled system 
j |-1 ©], [-te*=4 
r-[; 3) | se +5 


where y is defined by x = Ey. We can solve for y; and yg using normal first-order meth- 


yi ty = —2e% — 4 Yo + 38y2 = 3e% +5 
(ey) = —IJe* _ Ae’ (Gare = BY af Bett 
oe gt a ee C 3t _ 3 5 5 3¢ C 
Y= 3° e+, en yn = Fe ge se 
2 3 +) 
y= —3e" —44+Cye* y2 = al tat Cre" 


Therefore, our solution for y is 


and by converting back to 7, we get 


1 1 | ns —4+ no 


tt) = Ey= fe <7 Be 4 54 Cet 


_ | ge%®- 14+ Cie *+ Cre * 
et Ce = Ce "| 
15 3 
Or, rewriting in a different way, 


a= i et 4 Fi +O |) eae: ] en 8t, (4.13) 


15 3 


Notice how the general solution to the homogeneous equation (4.12) shows up at the 
end of this expression. 


2. Undetermined coefficients. Since the non-homogeneous part of our equation has terms 
of the form e” and constants, we should make a guess of the form 


Z,(t) = Be + D. 
We can plug this into our equation to get that 
eRe" (4.14) 


and the right hand side of the equation is 


—§ 5) fou. a, [era 
Pe 1 | (Be +B) + (BT | 
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Writing out Band D in components will give the right-hand side as 


= _ — = 2t 
5b, | et ahs 5d, | is k + 


4b, + bo 4d, + dy er" +3 
— |—5b; — 2b +1] » —5d, — 2d,+1 
| 4b) + bo +1 4d, +d2+3 


We can now set this equal to the left-hand side in (4.14) to get the vector equation 


2b, Ba —5b; —2bo +1] » =hdi — 2de 41 
Qbo | Aby bo +1 4d, +d,+3 


and we can match up the terms on the left and right sides to get a system that we need 
to solve: 
2b, = —5b,; — 2b2 +1 
2Qby = 4b, +b. +1 
0 = —5d, — 2d.+ 1 
0 = 4d, + dp + 3. 


Let’s start with the b equations. Rearranging these gives 


7b, + 2b5 = 1 — 4b; +b, =1 
Subtracting two copies of the second equation from the first gives 15b; = —1 or 
b; = —1/15, which gives by = 1+ 7c = iz. Next, we can solve the d equations, which we 


can rearrange to give 
5d + 2d2 = 1 4d, +d, =-3 


Subtracting two copies of the second equation from the first gives —3d, = 7 so d; = —7/s, 
leading to dy = —3 — 4(—7/3) = 19/3. Therefore, a solution to the non-homogeneous 
problem is 


—i ast 
Z(t) = i] ert fi 
15 3 
and so we can add in the homogeneous solution from (4.12) to get the full general 


solution as 

ul: i 1 1 

23 et = is + Ci e? + C2 et (4.15) 

B 3 —2 —1 

3. Variation of Parameters. For this method, we write down the fundamental matrix X(t) 
by combining the two basis solutions into a matrix, as 


xO=|$e4 Ss 


and compute the inverse matrix as 
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We can then work out the components of the method of variation of parameters. 


a~@ .l—e =e | fe*441 
X(t) oF — Pe =| ie ac 
= git _. gt pt — Sat 
~ ct ae Dee" 4h ap™ oe so 


_ [—2e% — 4e? 
~ | Bem + be% | * 


Integrating this expression gives 


4e° +C 
[xo lf dt = rea 5t eee 


and so the general solution to this system is 
sa er @ de’ +C 
xi fx “f a= i | nar 5 ao. 


_ —4+C,e* + & Beet +3 + Coe—*# 
7 re AL OC eS se (oe 


= aie, = i + Cye7§ + Gee 


(4.16) 


Bee = We = Oe 


Notice again that the homogeneous solution (4.12) shows up at the end of these terms, 
so we do not need to add it in at the end. 


Comparing the solutions (4.13), (4.15), and (4.16), we see that the three solutions generated 
by these three methods are all the same. | 


For this previous example, we only found the general solution. If the solution to an initial 
value problem was needed, we would need to wait until the very end, once we have figured out 
the solution to the non-homogeneous problem and added in the solution to the homogeneous 
problem to determine the value of the constants to meet the initial condition. 


4.8.3 Exercises 

Exercise 4.8.5: Find a particular solution to x’ = x + 2y + 2t, y’ = 3x + 2y — 4, 
a) using diagonalization, b) using undetermined coefficients. 

Exercise 4.8.6:* Find a particular solution to #’ = 5a+4y+t, y =x2+8y—t, 
a) using diagonalization, b) using undetermined coefficients. 

Exercise 4.8.7: Find the general solution to x’ = 4x + y —1, y’ =x+4y—e’, 


a) using diagonalization, b) using undetermined coefficients. 
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Exercise 4.8.8:* Find a particular solution to x’ =y+e',y’=x+e, 


a) using diagonalization, b) using undetermined coefficients. 
: 2 a ; ; ; 1 2 
Exercise 4.8.9: Let A = 9 7 This matrix has eigenvectors 9 and 1: 


a) Find a fundamental matrix, V(t), for the system x’ = AZ. 


Gt 
b) Use variation of parameters to solve the non-homogeneous system x’ = AZ + i | ; 


c) If we used method of undetermined coefficients instead, what would be the appropriate 
guess for the form of the non-homogeneous solution? 


Exercise 4.8.10:* Solve x’, = 22 +t, x, = x,+t with initial conditions x1(0) = 1, x2(0) = 2, 
using diagonalization. 


> 


Exercise 4.8.11: For each of the following vector functions f(t), find the general solution 
to the system of differential equations given by 


> 


c= io ‘a E+ f(t) 


using any of the methods described in this section. Notice the similarities and differences 
between using these methods for different non-homogeneous parts. 


a) F(t) = i b) f(t) = a 7 c) ft) = EB 
d) ft) = F a7 || Piel it f) f(t) = ie 


Exercise 4.8.12: The variation of parameters method can also be applied to constant 
coefficient systems. Find the general solution of the system 


at. |B “Las, et 
z=() 3] 2+ | 


a) diagonalization b) variation of parameters. 


using 


Compare and contrast these methods. You can use undetermined coefficients to check your 
answer. 


Exercise 4.8.13: Find the general solution to the differential equation 


»  {[-3 -l],, fe*+1 
c= 4 3 + 9 ; 
The best option is undetermined coefficients here because of the eigenvalues of the matrix. 


Diagonalization can be used, but care will be needed with solving the decoupled system 
because the coefficients will be complex. 
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Exercise 4.8.14: Find the general solution to the differential equation 


4  |—-5 16], cos(2t) 
a ie 3} 77 |sin(2t) — 2cos(2#)| ° 
The best option is undetermined coefficients here because of the eigenvalues of the matrix. 
We can’t actually use diagonalization (try it and see why!). 
Exercise 4.8.15: Find the general solution to the differential equation 
»  [-2 —12 4 e+e 
od eae ee a 


Exercise 4.8.16: Consider the system 


dx dy 
ae Qy + 4e%t. —2 — 3¢ — e**. 4.1 
a z+ 2y + 4e"; Ai se (4.17) 


a) Rewrite (4.17) in the form Z = Az + g(t), where x’ = AZ is a homogeneous system, 
and g(t) is a vector-valued function. 


b) Solve (4.17) using Method of Undetermined Coefficients. 


Exercise 4.8.17: 


— s 1 —4] , rae 
a) Use variation of parameters to solve the system x" = 4 — i k 0 | ; 


b) What does that solution tell you about how to set up the guess for the method of 
undetermined coefficients when there is a repeated eigenvalue? 


Exercise 4.8.18: Solve the initial value problem 


eB stel) -L) 


Exercise 4.8.19: Solve the initial value problem 


Exercise 4.8.20: Solve the initial value problem 
|S 2) a eee 
=| af #0=[A) 


qe, 2 
Exercise 4.8.21: Take the equation 7 = i | x+ wl ; 


t 


a) Check that %. = cy ésint | : I cost 


—t cost t see is the complementary solution. 


b) Use variation of parameters to find a particular solution. 
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4.9 Second order systems and applications 


Attribution: [JL], §3.6. 


After this section, you will be able to: 


e Use second order systems to model physical problems and 


e Solve second order systems using diagonalization or eigenvalue methods. 


4.9.1 Undamped mass-spring systems 


While we did say that we will usually only look at first order systems, it is sometimes more 
convenient to study the system in the way it arises naturally. For example, suppose we have 
3 masses connected by springs between two walls. We could pick any higher number, and 
the math would be essentially the same, but for simplicity we pick 3 right now. Let us also 
assume no friction, that is, the system is undamped. The masses are m1, m2, and m3 and 
the spring constants are k,, ko, k3, and k4. Let x; be the displacement from rest position of 
the first mass, and x2 and x3 the displacement of the second and third mass. We make, as 
usual, positive values go right (as 2, grows, the first mass is moving right). See Figure 4.16. 


ky ko kz kg 


ma UMW? AWW 8 


Figure 4.16: System of masses and springs. 


This simple system turns up in unexpected places. For example, our world really consists 
of many small particles of matter interacting together. When we try the system above with 
many more masses, we obtain a good approximation to how an elastic material behaves. 

Let us set up the equations for the three mass system. By Hooke’s law, the force acting 
on the mass equals the spring compression times the spring constant. By Newton’s second 
law, force is mass times acceleration. So if we sum the forces acting on each mass, put the 
right sign in front of each term, depending on the direction in which it is acting, and set this 
equal to mass times the acceleration, we end up with the desired system of equations. 


mya, = —kya1 + ko(a2 — 21) = —(ky + kq)a, + kore, 
M2x5 = —ko(to — 41) + k3(a3 — 22) == koa — (ko + kg) to + kas, 
m3t3 = —k3(xg3 — £2) — kay = kgtq — (kg + ky) a3. 
We define the matrices 
m, 0 0 —(k, + ky) kp 0 
M= 0 ms 0 and K= ko —(k2 + k3) k3 


0 O ms 0 kg —(kg + ka) 
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We write the equation simply as 

Mz" = Kz. 
At this point we could introduce 3 new variables and write out a system of 6 first order 
equations. We claim this simple setup is easier to handle as a second order system. We call 7 
the displacement vector, M the mass matrix, and K the stiffness matrix. 


Exercise 4.9.1: Repeat this setup for 4 masses (find the matrices M and K ). Do it for 5 
masses. Can you find a prescription to do it for n masses? 


As with a single equation we want to “divide by M.” This means computing the inverse 
of M. The masses are all nonzero and M is a diagonal matrix, so computing the inverse is 
easy: 


a 0 0 
Ver NO) ne 
0° 0. 

m3 


This fact follows readily by how we multiply diagonal matrices. As an exercise, you should 
verify that MM-'= M3M =I. 


Let A= M-!K. We look at the system #” = M~' KZ, or 
x! = Az. 


Many real world systems can be modeled by this equation. For simplicity, we will only talk 
about the given masses-and-springs problem. We try a solution of the form 


= Te. 


We compute that for this guess, #” = a?ve™. We plug our guess into the equation and get 


ave = Ave™. 
We divide by e® to arrive at a? = Av. Hence if a? is an eigenvalue of A and @ is a 
corresponding eigenvector, we have found a solution. 

In our example, and in other common applications, A has only real negative eigenvalues 
(and possibly a zero eigenvalue). So we study only this case. When an eigenvalue A is negative, 
it means that a? = is negative. Hence there is some real number w such that —w? = . 
Then a = +iw. The solution we guessed was 


T= U (cos(wt) + isin(wt)). 


By taking the real and imaginary parts (note that v is real), we find that Ucos(wt) and 
Vsin(wt) are linearly independent solutions. 

If an eigenvalue is zero, it turns out that both v and vt are solutions, where v is an 
eigenvector corresponding to the eigenvalue 0. 


Exercise 4.9.2: Show that if A has a zero eigenvalue and U is a corresponding eigenvector, 
then % = v(a + bt) is a solution of £” = A for arbitrary constants a and b. 
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Theorem 4.9.1 


Let A be a real n X n matrix with n distinct real negative (or zero) eigenvalues we 
denote by —w? > —w? > --- > —w?, and corresponding eigenvectors by wi), U2, ..., Un- 
If A is invertible (that is, if w; > 0), then 


n 


= Sy (a; cos(w;t) + bj sin(w;t)), 


j= 
is the general solution of 
for some arbitrary constants a; and 0b;. If A has a zero eigenvalue, that is w; = 0, and 


all other eigenvalues are distinct and negative, then the general solution can be written 
as 


X(t) = vi (a, + byt) + Se J; (a; cos(w;t) + b; sin(wit)). 


1=2 


We use this solution and the setup from the introduction of this section even when some 
of the masses and springs are missing. For example, when there are only 2 masses and only 2 
springs, simply take only the equations for the two masses and set all the spring constants 
for the springs that are missing to zero. 


4.9.2 Examples 


Example 4.9.1: Consider the setup in Figure 4.17, with m; = 2kg, my = 1kg, ky = 4N/m, 
and ko = 2N/m. 


ky kg 


ms HNL” 


Figure 4.17: System of masses and springs. 


Solution: The equations we write down are 


dep? 


or 


We find the eigenvalues of A to be A = —1,-—4 (exercise). We find corresponding 


eigenvectors to be [4] and [ 1, ] respectively (exercise). 
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We check the theorem and note that w, = 1 and wy = 2. Hence the general solution is 
, i) . 1 . 
f= H (a; cos(t) + bi sin(t)) + ] (a2 cos(2t) + bz sin(2t)). 


The two terms in the solution represent the two so-called natural or normal modes of 
oscillation. And the two (angular) frequencies are the natural frequencies. The first natural 
frequency is 1, and second natural frequency is 2. The two modes are plotted in Figure 4.18. 


0.0 2.5 5.0 7.5 10.0 0.0 25 5.0 75 10.0 


Figure 4.18: The two modes of the mass-spring system. In the left plot the masses are moving 
in unison and in the right plot are masses moving in the opposite direction. 


Let us write the solution as 
3 1 1 
t= || C1 cos(t — ai) + _1| © cos(2t — ag). 


The first term, 
c, cos(t — ay) 


H Co = Fa cos(t — - : 


corresponds to the mode where the masses move synchronously in the same direction. 
The second term, 

1 mest ac C2 cos(2t — ag) 

=e 2) ~~ |e cos(2t — a2)}’ 


corresponds to the mode where the masses move synchronously but in opposite directions. 

The general solution is a combination of the two modes. That is, the initial conditions 
determine the amplitude and phase shift of each mode. As an example, suppose we have 
initial conditions 
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We use the a;,b; constants to solve for initial conditions. First 


1 _ = _ 1 1 — | a + a2 
== bla Ale ee] 
We solve (exercise) to find a; = 0, ag = 1. To find the b; and bs, we differentiate first: 


= H (—a; sin(t) + b; cos(t)) + 7] (—2az sin(2t) + 2b2 cos(2t)). 


p]-cm—fos[- fea) 


Again solve (exercise) to find b; = 2, bg = —1. So our solution is 


Now we solve: 


z= [f] ance + [2] (ome — sana = Pan +n da) 


The graphs of the two displacements, 7; and x2 of the two carts is in Figure 4.19. 


Figure 4.19: Superposition of the two modes given the initial conditions. 


_| 


Example 4.9.2: We have two toy rail cars. Car 1 of mass 2kg is traveling at 3™/s towards 
the second rail car of mass 1 kg. There is a bumper on the second rail car that engages at the 
moment the cars hit (it connects to two cars) and does not let go. The bumper acts like a 
spring of spring constant k = 2N/m. The second car is 10 meters from a wall. See Figure 4.20 
on the next page. 

We want to ask several questions. At what time after the cars link does impact with the 
wall happen? What is the speed of car 2 when it hits the wall? 


Solution: OK, let us first set the system up. Let t = 0 be the time when the two cars link 
up. Let x; be the displacement of the first car from the position at t = 0, and let x2 be the 
displacement of the second car from its original location. Then the time when z2(t) = 10 is 
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k 
my mg 
oe WWie 5 


10 meters 


Figure 4.20: The crash of two rail cars. 


exactly the time when impact with wall occurs. For this t, 74(t) is the speed at impact. This 
system acts just like the system of the previous example but without k,. Hence the equation 


is 
210) 4 [2 213 
Gage ee soli 


gl = = 1 Zz 
g .29\" 
We compute the eigenvalues of A. It is not hard to see that the eigenvalues are 0 and —3 
1 


(exercise). Furthermore, eigenvectors are [+] and [ ',] respectively (exercise). Then w; = 0, 
Ww: = /3, and by the second part of the theorem the general solution is 


or 


z= H Gham |) («a2 cos( vB t) + by sin(V3#)) 


—_ [| ay + bit + ag cos(V3t) + be sin(V3 t) 
aa teh E = Das cos(/3 t) — 2bo sin(/3t) : 


We now apply the initial conditions. First the cars start at position 0 so 71(0) = 0 and 
x2(0) = 0. The first car is traveling at 3™/s, so x(0) = 3 and the second car starts at rest, so 
x(0) = 0. The first conditions says 


ait — | a + a2 
0 = £(0) = i - | ; 
It is not hard to see that a; = aj = 0. We set a; = 0 and ag = 0 in Z(t) and differentiate to 


get 
#(t) = i + 3b. cos(/3t) | 
|b — 23 be cos(V/3 t) | 


i ~#(0) = hee dl | 


Solving these two equations we find b; = 2 and by = Fe Hence the position of our cars is 
(until the impact with the wall) 
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Note how the presence of the zero eigenvalue resulted in a term containing t. This means that 
the cars will be traveling in the positive direction as time grows, which is what we expect. 

What we are really interested in is the second expression, the one for 72. We have 
L(t) = 2t — aR sin(/3t). See Figure 4.21 for the plot of x2 versus time. 

Just from the graph we can see that time of impact will be a little more than 5 seconds 
from time zero. For this we have to solve the equation 10 = x(t) = 2t — a sin(V3t). Using 
a computer (or even a graphing calculator) we find that timpact © 5.22 seconds. 

The speed of the second car is 24 = 
2 —2cos(V3t). At the time of impact (5.22 
seconds from t = 0) we get 24(timpact) 3.85. 
The maximum speed is the maximum of 
2 —2cos(V3t), which is 4. We are traveling 
at almost the maximum speed when we hit 75 
the wall. 


10.0 


Suppose that Bob is a tiny person sitting 
on car 2. Bob has a Martini in his hand and - 
would like not to spill it. Let us suppose Bob 
would not spill his Martini when the first car 00 
links up with car 2, but if car 2 hits the wall ' , ° 
at any speed greater than zero, Bob will spill | Figure 4.21: Position of the second car in time 
his drink. Suppose Bob can move car 2afew | (ignoring the wall). 
meters towards or away from the wall (he 
cannot go all the way to the wall, nor can he 
get out of the way of the first car). Is there a “safe” distance for him to be at? A distance 
such that the impact with the wall is at zero speed? 

The answer is yes. Looking at Figure 4.21, we note the “plateau” between t = 3 and 
t =4. There is a point where the speed is zero. To find it we solve x}(t) = 0. This is when 
cos(/3t) = 1 or in other words when t = * wor ... and so on. We plug in the first value to 


J 0.0 


obtain x2 (25) = we = 7.26. So a “safe” distance is about 7 and a quarter meters from the 


wall. 
Alternatively Bob could move away from the wall towards the incoming car 2, where 
an) _ 8 


another safe distance is x ( +n) we a 14.51 and so on. We can use all the different t such 


that x5(t) = 0. Of course t = 0 is also a solution, corresponding to xr = 0, but that means 
standing right at the wall. _| 


4.9.3. Forced oscillations 
Finally we move to forced oscillations. Suppose that now our system is 
i" = Ad + F cos(wt). (4.18) 


That is, we are adding periodic forcing to the system in the direction of the vector F. 
As before, this system just requires us to find one particular solution Z,, add it to the 
general solution of the associated homogeneous system 7,, and we will have the general 
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solution to (4.18). Let us suppose that w is not one of the natural frequencies of 7” = AZ, 
then we can guess 
Ep = Ccos(wt), 


where € is an unknown constant vector. Note that we do not need to use sine since there 
are only second derivatives. We solve for ¢ to find z,. This is really just the method of 
undetermined coefficients for systems. Let us differentiate Z,, twice to get 


x = —wécos(wt). 


Plug Zp and £} into equation (4.18): 


= - 
Tp Axp 


————— = 
—wécos(wt) = A@cos(wt) +F cos(wt). 


We cancel out the cosine and rearrange the equation to obtain 


> 


(A+u*Né=—F. 
So ce 
@=(A+u"l) (-F). 
Of course this is possible only if (A + w?/) = (A — (—w*)J) is invertible. That matrix is 
invertible if and only if —w? is not an eigenvalue of A. That is true if and only if w is not a 
natural frequency of the system. 


We simplified things a little bit. If we wish to have the forcing term to be in the units of 
force, say Newtons, then we must write 


Mi" = K#+ Gcos(ut). 
If we then write things in terms of A = M~'K, we have 
Z" =M"Ki+M'Geos(wt) or 2” = A# + Fcos(wt), 


where F — M-'!G. 


Example 4.9.3: Let us take the example in Figure 4.17 on page 348 with the same param- 
eters as before: m, = 2, mg = 1, ky = 4, and ky = 2. Now suppose that there is a force 
2 cos(3t) acting on the second cart. 


Solution: The equation is 


B waft 2 Jes [oom of w= [53 Les [9 cont 


We solved the associated homogeneous equation before and found the complementary solution 
to be 


i,.= A (a; cos(t) + b; sin(t)) + ] (a2 cos(2t) + by sin(2t)). 
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The natural frequencies are 1 and 2. As 3 is not a natural frequency, we try @cos(3t). We 


invert (A + 37/): 
oe en oa ee ee 
+377} = | P| 
: 25 7 a 


i a Ss | 0 7 
seasenen=[& ][°)-[4] 
20 20 10 
Combining with the general solution of the associated homogeneous problem, we get that 
the general solution to #” = AZ + F'cos(wt) is 


Hence, 


i 
20 
=3 
10 


(az cos(2t) + by sin(2t)) + cos(3t). 


1 
CS fete, = ; (a; cos(t) + b; sin(t)) + 


We then solve for the constants a1, a2, 61, and bg using any initial conditions we are given. | 


Note that given force i. we write the equation as MZ” = K#+ f to get the units right. 
Then we write 2” = M"'K#@+M~-'f. The term g = M~'f in #” = AZ + @ is in units of 
force per unit mass. 

If w is a natural frequency of the system, resonance may occur, because we will have to 
try a particular solution of the form 


Z, = Etsin(wt) + d cos(wt). 


That is assuming that the eigenvalues of the coefficient matrix are distinct. Next, note that 
the amplitude of this solution grows without bound as t grows. 


4.9.4 Non-Homogeneous Solutions 


Undetermined coefficients 


Let the equation be 7 
f" = Ax + Fit), 


where A is a constant matrix. If F(t) is of the form Fi cos(wt), then as two derivatives of 
cosine is again cosine we can try a solution of the form 


Ex ECOs wt), 


and we do not need to introduce sines. 

If the F is a sum of cosines, note that we still have the superposition principle. If F’ a= 
Fi cos(wot) + F, cos(wit), then we would try @cos(wot) for the problem Z” = AZ + Fy cos(wot), 
and we would try bcos(w;t) for the problem #” = Az + F,cos(w;t). Then we sum the 
solutions. 

However, if there is duplication with the complementary solution, or the equation is of 
the form Z” = Ai’ + B#+ F(t), then we need to do the same thing as we do for first order 
systems. 
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You will never go wrong with putting in more terms than needed into your guess. You 
will find that the extra coefficients will turn out to be zero. But it is useful to save some 
time and effort. 


Eigenvector decomposition 


If we have the system 7 
z" = Az + f(t), 


we can do ezgenvector decomposition, just like for first order systems. 
Let Ay, A2,...,A, be the eigenvalues and Vv}, v,...,U, be eigenvectors. Again form the 
matrix H=[v; U2 --- U,]. Write 


E(t) = UV Er (t) + Ue Eo(t) +--+ + ty En(d). 
Decompose Zz in terms of the eigenvectors 
F(t) = gilt) + Be golt) +--+ +H In(O), 


where, again, g = E~'f. 
We plug in, and as before we obtain 


~ 


z” AE i 
Vey + Uoky +--+ tn, = A (WE + Dabo + +--+ GE) + igi + Voge + +++ + tnGn 
= Avy Ei + Aves +--- + Atnn + Uign + Voge +--- + UnGn 
= U1A181 + Uor28e + +++ + GnAnEn + U1 gi + U2 g2++-++ Un Gn 
= Uy (Agi 91) Wal Agee Ga) = Ol AnGy F On): 


We identify the coefficients of the eigenvectors to get the equations 


1=MGa +H, 
2 = Ack. + G2, 


Each one of these equations is independent of the others. We solve each equation using the 
methods of chapter 2. We write Z(t) = U1&1(t) + Uo€o(t) +--+ + UnEn(t), and we are done; 
we have a particular solution. We find the general solutions for €; through €,, and again 
E(t) = Vy E,(t) + GH &o(t) +--+ + G,€,(t) is the general solution (and not just a particular 
solution). 


Example 4.9.4: Let us do the same example from before using this method. 


Solution: The equation is 
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The eigenvalues are —1 and —4, with eigenvectors [4] and [_~,]. Therefore # = [5 1,] and 
E+ =4[34,]. Therefore, 


eae Lk A 0 _ = cos(3t) 
=Ef(t)= 3 ; ‘ , 0 7 E on | 


3 


1 


92 


So after the whole song and dance of plugging in, the equations we get are 
" 2 " 2 
==. 3 cos(3t), & = —4&2 — 3 cos(3t). 


For each equation we use the method of undetermined coefficients. We try C, cos(3t) for the 
first equation and C2 cos(3t) for the second equation. We plug in to get 


2 
—9C} cos(3t) = —C) cos(3t) + 3 cos(3t), 
2 
—9C2 cos(3t) = —4C2 cos(3t) — 3 cos(3t). 


We solve each of these equations separately. We get —9C = —C,+2/3 and —9C, = —4C, —2/s. 


And hence C, = ~!/12 and Cy = 2/15. So our particular solution is 
= |i) f=1 1 2 | t20 
c= H (= cos(3) + ee & cos(3) = Pa cos(3t). 
This solution matches what we got previously. | 


4.9.5 Exercises 


Exercise 4.9.3: Find a particular solution to 


7 0 0 Cos 
Exercise 4.9.4:* Find the general solution to E 2 3 ge" = | 2-4 0 E+ |. 


Exercise 4.9.5 (challenging): Let us take the example in Figure 4.17 on page 348 with the 
same parameters as before: m, = 2, kj = 4, and ky = 2, except for m2, which is unknown. 
Suppose that there is a force cos(5t) acting on the first mass. Find an m2 such that there 
exists a particular solution where the first mass does not move. 

Note: This idea is called dynamic damping. In practice there will be a small amount of 
damping and so any transient solution will disappear and after long enough time, the first 
mass will always come to a stop. 


Exercise 4.9.6: Let us take the Example 4.9.2 on page 350, but that at time of impact, car 
2 is moving to the left at the speed of 3™/s. 
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a) Find the behavior of the system after linkup. 


b) Will the second car hit the wall, or will it be moving away from the wall as time goes 
on? 


c) At what speed would the first car have to be traveling for the system to essentially stay 
in place after linkup? 


Exercise 4.9.7: Let us take the example in Figure 4.17 on page 348 with parameters 
my, = My, = 1, ky = kp = 1. Does there exist a set of initial conditions for which the first cart 
moves but the second cart does not? If so, find those conditions. If not, argue why not. 


Exercise 4.9.8:* Suppose there are three carts of equal mass m and connected by two 
springs of constant k (and no connections to walls). Set up the system and find its general 
solution. 


Exercise 4.9.9:* Suppose a cart of mass 2kg is attached by a spring of constant k = 1 to a 
cart of mass 3kg, which is attached to the wall by a spring also of constant k = 1. Suppose 
that the initial position of the first cart is 1 meter in the positive direction from the rest 
position, and the second mass starts at the rest position. The masses are not moving and are 
let go. Find the position of the second mass as a function of time. 


Exercise 4.9.10: Find the general solution to xf = —6x; + 342 + cos(t), #3 = 2x, — Tro + 
3cos(t), 


a) using eigenvector decomposition, b) using undetermined coefficients. 


Exercise 4.9.11: Find the general solution to x/ = —6x, + 3x2 + cos(2t), 75 = 2x”, — 7x2 + 
3.cos(2t), 


a) using eigenvector decomposition, b) using undetermined coefficients. 


Exercise 4.9.12:* Solve x = —32,+%9 +t, x} = 9x,+ 52+ cos(t) with initial conditions 
x1(0) = 0, r2(0) = 0, x (0) = 0, x4(0) = 0, using eigenvector decomposition. 
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4.10 Matrix exponentials 


Attribution: [JL], §3.8. 


After this section, you will be able to: 


e Compute the exponential of a matrix and 


e Use the matrix exponential to solve linear systems of differential equations. 


4.10.1 Definition 


There is another way of finding a fundamental matrix solution of a system. Consider the 
constant coefficient equation 
f= Ps. 


If this would be just one equation (when P is a number or a 1 x 1 matrix), then the solution 
would be 

#=eF*, 
That doesn’t make sense if P is a larger matrix, but essentially the same computation that 
led to the above works for matrices when we define e”* properly. First let us write down the 
Taylor series for e“ for some number a: 


t)? , (at)® . (at)" = (at)! 

at 7 t (a ee . 

€ eG 5 oe er > ‘i 

Recall k! = 1-2-3---k is the factorial, and 0! = 1. We differentiate this series term by term 
ae oy ee (at)” — (at)® : 
ate) SO te ee ala oe oe Pee 


Maybe we can try the same trick with matrices. 
Definition 4.10.1 


For an n X n matrix A we define the matrix exponential as 


e il 1 i 
Be ere ae | ek 


Let us not worry about convergence. The series really does always converge. We usually 
write Pt as tP by convention when P is a matrix. With this small change and by the exact 
same calculation as above we have that 
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Now P and hence e”” is an n X n matrix. What we are looking for is a vector. In the 1 x 1 
case we would at this point multiply by an arbitrary constant to get the general solution. In 


the matrix case we multiply by a column vector ¢. 
Theorem 4.10.1 


Let P be an n X n matrix. Then the general solution to 7’ = PZ is 


where C is an arbitrary constant vector. In fact, #(0) = ¢. 


Let us check: 


d d (tp tP 
—x= — (eC) = Pe" c= Pr. 
moe 
Hence e!” is a fundamental matrix solution of the homogeneous system. So if we can 


compute the matrix exponential, we have another method of solving constant coefficient 
homogeneous systems. It also makes it easy to solve for initial conditions. To solve z” = AZ, 
£(0) = b, we take the solution 


Z= ed. 


This equation follows because e°4 = I, so Z(0) = eb = b. 


We mention a drawback of matrix exponentials. In general e4+? 4 e4e?. The trouble is 
that matrices do not commute, that is, in general AB 4 BA. If you try to prove e4+? F e4e? 
using the Taylor series, you will see why the lack of commutativity becomes a problem. 
However, it is still true that if AB = BA, that is, if A and B commute, then e4+? = e4e?. 
We will find this fact useful. Let us restate this as a theorem to make a point. 


Theorem 4.10.2 


i Ap SA theme) — ce  “Othcrwicee 1 oee im teneral: 


4.10.2 Simple cases 


In some instances it may work to just plug into the series definition. Suppose the matrix is 
diagonal. For example, D = [6°]. Then 


and 


iL 1 
CLES ee 


1 0 a 0] 1 fa? 0 1fa? 0 e* 0 
=(5 + (6 ole vl tale alt= [5 HE 
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So by this rationale 


fav) 


jt e 0 al _ e* 0 
=|5 i and e€ =f a 


This makes exponentials of certain other matrices easy to compute. For example, the 
matrix A = [_*, {] can be written as 3. + B where B = [?, *,]. Notice that B? = [88]. So 
BY = 0 for all k > 2. Therefore, e? = J+ B. Suppose we actually want to compute e'4. The 
matrices 3tJ and tB commute (exercise: check this) and e'? = I +tB, since (tB)* = t?B? = 0. 
We write 


0 


7 i" a es At |- ee e = Ate! 


0 ef} | -t 1-2} — —te*t (1 — 2t) e**| 


tA 3tI+tB 3tI_ .tB er 1 
ec Ser Sere = e3t (I+tB) = 


We found a fundamental matrix solution for the system Z’ = AZ. Note that this matrix has 
a repeated eigenvalue with a defect; there is only one eigenvector for the eigenvalue 3. So we 
found a perhaps easier way to handle this case. In fact, if a matrix A is 2 x 2 and has an 
eigenvalue \ of multiplicity 2, then either A = \J, or A = \J + B where B? = 0. This is a 
good exercise. 


Exercise 4.10.1: Suppose that A is 2x2 and 2 is the only eigenvalue. Show that (A — I)? = 
0, and therefore that we can write A= AI + B, where B? = 0 (and possibly B = 0). Hint: 
First write down what does it mean for the eigenvalue to be of multiplicity 2. You will get an 
equation for the entries. Now compute the square of B. 


Matrices B such that B* = 0 for some k are called nilpotent. Computation of the matrix 
exponential for nilpotent matrices is easy by just writing down the first k terms of the Taylor 
series. 


4.10.3 General matrices 


In general, the exponential is not as easy to compute as above. We usually cannot write a 
matrix as a sum of commuting matrices where the exponential is simple for each one. But 
fear not, it is still not too difficult provided we can find enough eigenvectors. First we need 
the following interesting result about matrix exponentials. For two square matrices A and B, 
with B invertible, we have 


ePAB™ — BeAB-. 
This can be seen by writing down the Taylor series. First 


(BAB™!)? = BAB"'BAB™ = BAIAB™ = BA?B™. 
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And by the same reasoning (BAB-1)* — BA*B-!. Now write the Taylor series for e242": 
24 1 I! 
Ce BAR OS. (BAB) ce (BABS scoot: 
1 1 
= BB'+ BAB" + 5 BAB sf -BA’B™ eee 
1 1 


AP A’ +---)Bo 
2 6 


= B(UI+A4 
= Be4B-". 


Given a square matrix A, we can usually write A = EDE™', where D is diagonal and 
E invertible. This procedure is called diagonalization. If we can do that, the computation 
of the exponential becomes easy as e” is just taking the exponential of the entries on the 
diagonal. Adding t into the mix, we can then compute the exponential 


eS hel ns, 


To diagonalize A we need n linearly independent eigenvectors of A. Otherwise, this method 
of computing the exponential does not work and we need to be trickier, but we will not get 
into such details. Let E’ be the matrix with the eigenvectors as columns. Let A1, A2, .--, An 
be the eigenvalues and let v}, , ..., U, be the eigenvectors, then EF =[U, Ug +++ Uy]. 
Make a diagonal matrix D with the eigenvalues on the diagonal: 


Ker 20. Gla 20 
Gs i. See) 
0 O \n 
We compute 
AE=AlU, Ug -:: Un] 
=> [ Avi Avo Abn | 
=[Ait, Ag. +++ Antn| 
= [vi (0D) Up, |D 
= ED. 


The columns of F are linearly independent as these are linearly independent eigenvectors of 
A. Hence EF is invertible. Since AE = ED, we multiply on the right by E~! and we get 


A=EDE™. 


This means that e4 = Ee? E~!. Multiplying the matrix by t we obtain 


fA repo gl © | (4.19) 
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The formula (4.19), therefore, gives the formula for computing a fundamental matrix solution 
e'4 for the system z” = Az’, in the case where we have n linearly independent eigenvectors. 

This computation still works when the eigenvalues and eigenvectors are complex, though 
then you have to compute with complex numbers. It is clear from the definition that if A is 
real, then e’4 is real. So you will only need complex numbers in the computation and not 
for the result. You may need to apply Euler’s formula to simplify the result. If simplified 
properly, the final matrix will not have any complex numbers in it. 


Example 4.10.1: Compute a fundamental matrix solution using the matrix exponential for 


the system 
a] 1 2] |x 
y} [2 1 fy] 


Then compute the particular solution for the initial conditions x(0) = 4 and y(0) = 2. 
Let A be the coefficient matrix [}?]. We first compute (exercise) that the eigenvalues are 
3 and —1 and corresponding eigenvectors are [}] and [ +,]. Hence the diagonalization of A is 


Loi Ae Oy a 
Bed) 1 il -fO tier el), = 


We write 


I 
ro] 
rae 
| ra | 
Qa 
wow w 
aos 
lo 
@ I 
| + 

ow 
| a | 
| | 
a 
e | 
rae 
ss | 


—] ee figet oe on ae 


2 2 
= a 3t —t 3t —t 
et e€ t ext e€ t e e e-* +e 


2 2 


The initial conditions are x(0) = 4 and y(0) = 2. Hence, by the property that e°4 = I we 
find that the particular solution we are looking for is e'4b where b is [3]. Then the particular 


solution we are looking for is 


el. ate ae 4) [2e%* +2e*+e% —e*]  [3e% +e7* 
y) | ee | 2) ze" ee pepe) (setae * |" 


4.10.4 Fundamental matrix solutions 


We note that if you can compute a fundamental matrix solution in a different way, you can 
use this to find the matrix exponential e'4. A fundamental matrix solution of a system of 
ODEs is not unique. The exponential is the fundamental matrix solution with the property 


4.10. MATRIX EXPONENTIALS 363 


that for t = 0 we get the identity matrix. So we must find the right fundamental matrix 
solution. Let X be any fundamental matrix solution to 7’ = Az. Then we claim 


M4 = X(t) [XO]. 


Clearly, if we plug t = 0 into X(t) [X(0)]-* we get the identity. We can multiply a fundamental 
matrix solution on the right by any constant invertible matrix and we still get a fundamental 
matrix solution. All we are doing is changing what are the arbitrary constants in the general 
solution £(t) = X(t) é 


4.10.5 Approximations 


If you think about it, the computation of any fundamental matrix solution X using the 
eigenvalue method is just as difficult as the computation of e'4. So perhaps we did not gain 
much by this new tool. However, the Taylor series expansion actually gives us a way to 
approximate solutions, which the eigenvalue method did not. 

The simplest thing we can do is to just compute the series up to a certain number of 
terms. There are better ways to approximate the exponential*. In many cases however, few 
terms of the Taylor series give a reasonable approximation for the exponential and may suffice 
for the application. For example, let us compute the first 4 terms of the series for the matrix 
A= [33]. 

5 


2 
9 8 


a 


3 


iP - 
eI +taAt+ sA+cA=I+t a ee 


ab WIN 


6 
7 
21 u 


iw) 


14¢4+2?4+88 2¢42P+7268 
2¢4+274+728 144437438 


Just like the scalar version of the Taylor series approximation, the approximation will be 
better for small t and worse for larger t. For larger t, we will generally have to compute more 
terms. Let us see how we stack up against the real solution with t = 0.1. The approximate 
solution is approximately (rounded to 8 decimal places) 


0.1? 
AP a 


O.1A 
xI+0.1A4 
€ + 5 6 


0.1" 38 1.12716667 0.22233333 
~ |0.22233333 1.12716667| ° 


And plugging t = 0.1 into the real solution (rounded to 8 decimal places) we get 


014A _ {112734811 0.22251069 
~ -|0.22251069 1.12734811] ° 


Not bad at all! Although if we take the same approximation for t = 1 we get 


il 
fpAe_—f 4 
ue 


be = 6.66666667 6.33333333 
6 —_ |6.33333333 6.66666667 | ’ 


*C. Moler and C.F. Van Loan, Nineteen Dubious Ways to Compute the Exponential of a Matrix, Twenty- 
Five Years Later, SIAM Review 45 (1), 2003, 3-49 
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while the real value is (again rounded to 8 decimal places) 


A _ |10.22670818  9.85882874 
~ | 9.85882874 10.22670818] 


So the approximation is not very good once we get up to t = 1. To get a good approximation 
at t = 1 (say up to 2 decimal places) we would need to go up to the 11 power (exercise). 


4.10.6 Non-Homogeneous Systems 


Integrating factor 


Now that we have matrix exponentials, we can try to use them to help us solve non- 
homogeneous systems of differential equations. First, let’s recall what we did for first order 
equations. If we have an equation of the form 


x'(t) + pe(t) = f(t) 


where will assume that p is constant (even though it doesn’t have to be). We would go about 
solving this problem by multiplying both sides of the equation by e?', writing the left-hand 
side as a product rule, integrating both sides, and solving. 

With matrix exponentials, we can do exactly the same thing with first order systems. Let 
us focus on the nonhomogeneous first order equation 


z(t) = Az(t) + F(d), 


where A is a constant matrix. The method we look at here is the integrating factor method. 
For simplicity we rewrite the equation as 


z(t) + Pz(t) = f(t), 


where P = —A. We multiply both sides of the equation by e“” (being mindful that we are 
dealing with matrices that may not commute) to obtain 


eH (t) +  Pi(t) = et” f(t). 


We notice that Pe’? = e'? P. This fact follows by writing down the series definition of e’”: 


1 1 
PoP =P (I + tP 4 5(tP)° | --) =P+tP* + st P a 


1 
7 (1 + tP 4 5(tP) o) PaePP 


So £ (ce?) = Pe’ =e’? P. The product rule says 


© (Patt) = e #(t) + P(t), 
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and so 


> 


“ ('?z() = P F(t), 


We can now integrate. That is, we integrate each component of the vector separately 
eP z(t) = perio dt +@. 
In Exercise 4.10.10, you will compute and verify that (e'? he =e'P. Therefore, we obtain 
#th=e? | cP f(t)dtt+e Ve 


Perhaps it is better understood as a definite integral. In this case it will be easy to also 
solve for the initial conditions. Consider the equation with initial conditions 


=> 


i (t)+ P#(t)=f(t), 20) =6. 


The solution can then be written as 


t 
| a(t) = ae) e*? f(s) ds +e *d. (4.20) 
0 


Again, the integration means that each component of the vector e°? fi (s) is integrated 
separately. It is not hard to see that (4.20) really does satisfy the initial condition 7(0) = b. 


0 
70) = cor f e®? f(s)\ds +e °'b = Ib=b. 
0 


Example 4.10.2: Suppose that we have the system 


wv’, + 521 — 32. =e’, 
Lo + 3x1 — 22 = 0, 
with initial conditions 7,(0) = 1, x2(0) = 0. 


Solution: Let us write the system as 


eo Set: 0-6 


The matrix P = [3 —}] has a doubled eigenvalue 2 with defect 1, and we leave it as an 
exercise to double check we computed e“” correctly. Once we have e“”, we find e~"”, simply 
by negating tf. 


oP (1 + 3t) e# —fie* AP as (1 — 3t)e" Sie 
3te*" (1—3t)e*|’ —3te* (14 3t)e"™ | 
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Instead of computing the whole formula at once, let us do it in stages. First 
t t 2s 2s 8 
“PF = (1+ 3s)e —3se € 
/ dN / 3se75 (1 — 3s) e?°| | 0 o 
t a 38 
_ i, " 38) | ds 
0 3SE 


_ 5 (1 + 3s) e* ds 
i 3se°* ds 


tes 
= xyes | (used integration by parts). 
3 


Then 
t 
x) =e"? | et? F(s) ds +e 


0 
_ [ste  3te te™ , [- 3¢) eh aie i 
~ [| —3te* — (1+ 3t)e~**] [Ge Det ote: Che Bh) er (0 
7 te>* i (1 — 3t)e" 
7 a +(§+t) e* —3te—* 
= | (1 — 2t) e~*" | 


Ce a 


Phew! 
Let us check that this really works. 


w+ 5a, — 3a_ = (4te** — 4e-™) +. 51 — Qt) * +e! — (1 —6t)e * = et. 
Similarly (exercise) 75 + 321 — x2 = 0. The initial conditions are also satisfied (exercise). _| 


For systems, the integrating factor method only works if P does not depend on t, that is, 
P is constant. The problem is that in general 


< jel POe| 4 P(t) ef P(t) at. 


because matrix multiplication is not commutative. 


4.10.7 Exercises 


Exercise 4.10.2: Using the matrix exponential, find a fundamental matrix solution for the 
system v' = 3a +y, y =a 4+3y. 


Exercise 4.10.3: Find e'4 for the matrix A = [23]. 
Exercise 4.10.4:* Compute ce‘ where A= | !, 7]. 
Exercise 4.10.5:* Compute e'4 where A = | -2 7 2| 


Exercise 4.10.6:* 
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a) Compute e'4 where A = [351]. b) Solve g’ = Az for z(0) = (4). 
Exercise 4.10.7: Find a fundamental matrix solution for the system x, = 711 +472 + 1223, 
fy = 41+ 24.4 £3, V3 = —321 — 242 — 5x3. Then find the solution that satisfies £(0) = | 3] 
Exercise 4.10.8: Compute the matrix exponential e4 for A = [}?]. 


Exercise 4.10.9 (challenging): Suppose AB = BA. Show that under this assumption, 


cAtB — eAcB. 


Exercise 4.10.10: Use Exercise 4.10.9 to show that (eA)? =e~“. In particular this means 
that e4 is invertible even if A is not. 


Exercise 4.10.11: Let A be a 2 x 2 matrix with eigenvalues —1, 1, and corresponding 
eigenvectors [+], [9]. 


a) Find matrix A with these properties. 
b) Find a fundamental matrix solution to x’ = AZ. 
c) Solve the system in with initial conditions X(0) = [3] . 


Exercise 4.10.12: Suppose that A is an n X n matrix with a repeated eigenvalue » of 
multiplicity n. Suppose that there are n linearly independent eigenvectors. Show that the 
matrix is diagonal, in particular A = AI. Hint: Use diagonalization and the fact that the 
identity matrix commutes with every other matrix. 


Exercise 4.10.13: Let A = [4' 73]. 
a) Find e'4. b) Solve #’ = Az, £(0) = [ 4]. 


Exercise 4.10.14: Let A = [43]. Approximate e'4 by expanding the power series up to the 
third order. 


Exercise 4.10.15:* Compute the first 3 terms (up to the second degree) of the Taylor 


expansion of e'4 where A = [33] (Write as a single matrix). Then use it to approximate 
0.1A 
o 


Exercise 4.10.16: For any positive integer n, find a formula (or a recipe) for A" for the 
following matrices: 


3.0 5 2 0 1 2 1 
ol Pt ob afd 
Exercise 4.10.17:* For any positive integer n, find a formula (or a recipe) for A” for the 
following matrices: 


o {3 » [5 S| Jl | 
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Exercise 4.10.18: Solve the initial value problem 


-Balel] --[ 


using matrix exponentials. 


Exercise 4.10.19: Solve the initial value problem 


using matrix exponentials. 


Exercise 4.10.20: Solve the initial value problem 


using matrix exponentials. 


Chapter 5 


Nonlinear systems 


5.1 Linearization, critical points, and stability 


Attribution: [JL], §8.1, 8.2. 


After this section, you will be able to: 


e Find critical points of a non-linear system of differential equations, 

Linearize a non-linear system around a critical point, 

Determine if a critical point of a non-linear system is isolated, 

Use the Jacobian matrix to classify the critical point of a non-linear system, and 


Determine the stability of a critical point from the classification. 


Except for a few brief detours in chapter 1, we considered mostly linear equations. Linear 
equations suffice in many applications, but in reality most phenomena require nonlinear 
equations. Nonlinear equations, however, are notoriously more difficult to understand than 
linear ones, and many strange new phenomena appear when we allow our equations to be 
nonlinear. 

Not to worry, we did not waste all this time studying linear equations. Nonlinear equations 
can often be approximated by linear ones if we only need a solution “locally,” for example, 
only for a short period of time, or only for certain parameters. Understanding specific linear 
equations can also give us qualitative understanding about a more general nonlinear problem. 
The idea is similar to what you did in calculus in trying to approximate a function by a line 
with the right slope. 

In § 2.4 we looked at the pendulum of length L. The goal was to 


solve for the angle @(t) as a function of the time t. The equation for 
the setup is the nonlinear equation L 
g 6 
6” + — sind = 0. 
L 
m 


Instead of solving this equation, we solved the rather easier linear 
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equation 
WV g 
7 ~9=0. 
L 


While the solution to the linear equation is not exactly what we were looking for, it is rather 
close to the original, as long as the angle @ is small and the time period involved is short. 

You might ask: Why don’t we just solve the nonlinear problem? Well, it might be 
very difficult, impractical, or impossible to solve analytically, depending on the equation in 
question. We may not even be interested in the actual solution, we might only be interested 
in some qualitative idea of what the solution is doing. For example, what happens as time 
goes to infinity? 


5.1.1 Autonomous systems and phase plane analysis 


We restrict our attention to a two-dimensional autonomous system 


v=f(a,y), y =9(2,y), 


where f(x,y) and g(x,y) are functions of two variables, and the derivatives are taken with 
respect to time t. Solutions are functions x(t) and y(t) such that 


x'(t)=f(2(t),y@), y(t) = (x(t), y(@). 


The way we will analyze the system is very similar to § 1.7, where we studied a single 
autonomous equation. The ideas in two dimensions are the same, but the behavior can be 
far more complicated. 

It may be best to think of the system of equations as the single vector equation 


| - pe 4 (5.1) 


y g(x, y) 


As in § 4.1 we draw the phase portrait (or phase diagram), where each point (x, y) corresponds 
to a specific state of the system. We draw the vector field given at each point (x,y) by the 


f(x,y) 
g(x,y) 


points (x(t), y(t)) for a certain range of t. 


vector . And as before if we find solutions, we draw the trajectories by plotting all 


Example 5.1.1: Consider the second order equation x” = —x + x”. Write this equation as 
a first order nonlinear system 


zg =y, y = —24+ 2". 
The phase portrait with some trajectories is drawn in Figure 5.1 on the facing page. 

From the phase portrait it should be clear that even this simple system has fairly 
complicated behavior. Some trajectories keep oscillating around the origin, and some go off 
towards infinity. We will return to this example often, and analyze it completely in this (and 
the next) section. 


If we zoom into the diagram near a point where E | is not zero, then nearby the arrows 


point generally in essentially that same direction and have essentially the same magnitude. In 


5.1. LINEARIZATION, CRITICAL POINTS, AND STABILITY 371 


Ss sasasnawrrr® 


ae 
a 
f 
4 
t 
t 
t 
t 
t 
t 
t 
4 
‘ 
‘ 
‘ 
: 
‘ 
\ 
x 
\ 


AAA ADAP PFPPPP Ss sarararss 


2 - 
2 =) 0 1 2 


Figure 5.1: Phase portrait with some trajectories of x! = y, y! = —x +27. 


other words the behavior is not that interesting near such a point. We are of course assuming 
that f(x,y) and g(x,y) are continuous. 

Let us concentrate on those points in the phase diagram above where the trajectories 
seem to start, end, or go around. We see two such points: (0,0) and (1,0). The trajectories 
seem to go around the point (0,0), and they seem to either go in or out of the point (1,0). 
These points are precisely those points where the derivatives of both x and y are zero. 


Definition 5.1.1 


The critical points of a system of differential equations 


dx 
dt 
dy 
dt 


en] =" 


In other words, these are the points where both f(x,y) = 0 and g(z, y) = 0. 


= f(x,y) 


= g(x,y) 


are the points (x,y) such that 


The critical points are where the behavior of the system is in some sense the most 


complicated. If E | is zero, then nearby, the vector can point in any direction whatsoever. 


Also, the trajectories are either going towards, away from, or around these points, so if we are 
looking for long-term qualitative behavior of the system, we should look at what is happening 
near the critical points. 

Critical points are also sometimes called equilibria, since we have so-called equilibrium 
solutions at critical points. If (xo, yo) is a critical point, then we have the solutions 


x(t)=2, y(t) = yo. 
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In Example 5.1.1 on page 370, there are two equilibrium solutions: 
e) =O. =U; aid @)=1, o=—t 


The discussion here should seem a bit familiar; it is the same as how we formulated equilibrium 
solutions to autonomous differential equations in in § 1.7. 


5.1.2 Linearization 


How do linear systems fit into this approach? For a linear, homogeneous system of two 
variables defined by 
Zz = At 

where A is an invertible matrix, the only critical point is the origin (0,0). Since A is invertible, 
the only vector that satisfies AZ = 0 is 7 = 0, see § 3.5. (This also applies beyond two variables, 
but we’ll stick to that for simplicity.) In § 4.7 we studied the behavior of a homogeneous 
linear system of two equations near a critical point. Let us put the understanding we gained 
in that section to good use understanding what happens near critical points of nonlinear 
systems. 

In calculus we learned to estimate a function by taking its derivative and linearizing. We 
work similarly with nonlinear systems of ODE. Suppose (29, yo) is a critical point. In order 
to linearize the system of differential equations, we want to linearize the two functions f(z, y) 
and g(x,y) that define this system. To do so, we will replace f and g by the tangent plane 
approximation to the functions. That is, if we set z = f(x,y), the tangent plane is given by 


L;(x,y) = f(0, Yo) + fe(Xo, Yo) (x — Lo) + fy(Xo, Yo)(Y — Yo). 


Since (20, yo) is a critical point, we know that f(x, yo) = 0, so the tangent plane is given by 


Ls (x,y) = fe(®0, Yo)(@ — Lo) + fy(Xo, Yo)(Y — Yo). 


Similarly, the tangent plane for g(x,y) near the critical point (9, yo) is given by 


Lg(x,y) = gx(Xo, Yo)(X — Lo) + Jy(Xo, Yo)(Y — Yo): 


The idea of linearization in calculus was that we could use the tangent line or tangent 
plane to approximate a function near to a given point. For systems of differential equations, 
the idea is that we can approximate the solutions to the system of differential equations by 
the solutions to the linearized systems as long as we stay near the critical point. That means 
that we can approximate the solution to 


ax 
dy 
Ho Ie) 
near the critical point (29, yo) by the solution to the system 
dx 
a fic(X0, Yo)(@ — Lo) + fy(Xo, Yo)(Y — Yo) 
dy 


a = Gx (Xo, Yo) (x _— Xo) + Jy (Xo, yo) (y _ Yo) 
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Next, change variables to (u,v), so that (u,v) = (0,0) corresponds to (xo, yo). That is, 
U= 2X — Xo, Vv=Y— Yo, 


which is not going to affect our differential equations because x and yo are constant. 


Since a = a and dy = we can rewrite the approximation system as 
du 
a feo, Yo)U + fy(Lo, Yo)v 
dv 
ra Jx(Lo, Yo)U + Gy(Lo, Yo)Y 
In multivariable calculus you may have seen that the several variables version of the 


| 


derivative is the Jacobian matriz*. The Jacobian matrix of the vector-valued function fe 


at (9, yo) is 


(29, yo) FE (x0, Yo) 
a a 

5a (Lo, Yo) By (X05 yo) 

This matrix gives the best linear approximation as u and v (and therefore x and y) vary. 


Definition 5.1.2 


The linearization of the equation (5.1) as the linear system 


a (Xo, Yo) 
“8x9, yo) 


Example 5.1.2: Determine the linearization of the system of differential equations in Ex- 
ample 5.1.1: 2’ = y, y’ = —x +2” at all of its critical points. 


Solution: There are two critical points, (0,0) and (1,0). The Jacobian matrix at any point 


: [ae ger) =| 0 Ap 


Aa (x,y) $2(2,y) —1+2r 0 


Therefore at (0,0), we have u= x and v = y, and the linearization is 


P-L Jy 


At the point (1,0), we have u = x — 1 and v = y, and the linearization is 


-b df 


The phase diagrams of the two linearizations at the point (0,0) and (1,0) are given 
in Figure 5.2 on the following page. Note that the variables are now u and v. Compare 
Figure 5.2 with Figure 5.1 on page 371, and look especially at the behavior near the critical 


points. | 
*Named for the German mathematician Carl Gustav Jacob Jacobi (1804-1851). 
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Figure 5.2: Phase diagram with some trajectories of linearizations at the critical points (0,0) 
(left) and (1,0) (right) of x’ =y, y =—-x+4+ 27. 


5.1.3 Isolated critical points and almost linear systems 


The next step in this process is to try to figure out a way to analyze what is happening to a 
non-linear system of differential equations near equilibrium solutions without using a slope 
field/phase portrait. We would like to be able to determine this from the equations alone, not 
any of the pictures that come from them. Thankfully, our ability to analyze linear systems 
helps us accomplish this goal. 


Definition 5.1.3 


A critical point is isolated if it is the only critical point in some small “neighborhood” 
of the point. 


That is, if we zoom in far enough it is the only critical point we see. In the example 
above, the critical point was isolated. If on the other hand there would be a whole curve of 
critical points, then it would not be isolated. For example, the system 


w=ya@-l) y= («-2)(4—-1) 


has the entire line x = 1 as critical points. Therefore, these are not isolated. 
Definition 5.1.4 


A system is called almost linear at a critical point (29, yo), if the critical point is isolated 


and the Jacobian matrix at the point is invertible, or equivalently if the linearized 
system has an isolated critical point. 


This is also equivalent to zero not being an eigenvalue of the Jacobian matrix at the 
critical point. In such a case, the nonlinear terms are very small and the system behaves like 
its linearization, at least if we are close to the critical point. 
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For example, the system in Examples 5.1.1 and 5.1.2 has two isolated critical points (0, 0) 
and (0,1), and is almost linear at both critical points as the Jacobian matrices at both points, 


[ ©; 6] and [9 4], are invertible. 


On the other hand, the system x’ = 2”, y' = y” has an isolated critical point at (0,0), 
however the Jacobian matrix 
2x 0 
O 2y 


is zero when (x,y) = (0,0). So the system is not almost linear. Even a worse example is the 
system x’ = x, y’ = x”, which does not have isolated critical points; x’ and y’ are both zero 
whenever x = 0, that is, the entire y-axis. 


Fortunately, most often critical points are isolated, and the system is almost linear at the 
critical points. So if we learn what happens there, we will have figured out the majority of 
situations that arise in applications. 


5.1.4 Stability and classification of isolated critical points 


Once we have an isolated critical point, the system is almost linear at that critical point, and 
we computed the associated linearized system, we can classify what happens to the solutions. 
The classifications for linear two-variable systems from § 4.7 are generally the same as what 
we use here, with one minor caveat. Let us list the behaviors depending on the eigenvalues of 
the Jacobian matrix at the critical point in Table 5.1. This table is very similar to Table 4.1 
on page 324, with the exception of missing “center” points. The repeated eigenvalue cases 
are also missing. They behave similarly to the real eigenvalue descriptions in the table below, 
but similar to centers, the behavior can change slightly. It can behave like either a spiral or a 
node, but will be either a source or sink based on the sign of the repeated eigenvalue. We 
will discuss centers later, as they are more complicated. 


Eigenvalues of the Jacobian matrix Behavior Stability 
real and both positive source / unstable node unstable 
real and both negative sink / stable node asymptotically stable 
real and opposite signs saddle unstable 
complex with positive real part spiral source unstable 
complex with negative real part spiral sink asymptotically stable 


Table 5.1: Behavior of an almost linear system near an isolated critical point. 


In the third column, we mark points as asymptotically stable or unstable. 


376 CHAPTER 5. NONLINEAR SYSTEMS 


Definition 5.1.5 


Let (xo, yo) be a critical point for a non-linear system of two differential equations. 


1. We say that the critical point is a stable critical point if, given any small distance 
€ to (20, yo), and any initial condition within a perhaps smaller radius around 
(Xo, Yo), the trajectory of the system never goes further away from (29, yo) than e. 


. The critical point is an unstable critical point if it is not stable; that is, there are 
trajectories that start within a distance e€ of (xo, yo) and end up farther than e€ 
from that point. 


. The critical point is called asymptotically stable if given any initial condition 
sufficiently close to (xo, yo) and any solution (a(t), y(t)) satisfying that condition, 
then 


lim (x(t), y(t)) = (0, Yo). 


too 


Informally, a point is stable if we start close to a critical point and follow a trajectory we 
either go towards, or at least not away from, this critical point. If the point is asymptotically 
stable, then any trajectory for a sufficiently close initial condition goes towards the critical 
point (29, Yo), and unstable means that, in general, trajectories move away from the critical 
point. 


Example 5.1.3: Find and analyze the critical points of 7’ = —y — x7, y! =—xv+y?. 


Solution: See Figure 5.3 for the phase diagram. Let us find the critical points. These are 
the points where —y — x? = 0 and —x + y? = 0. The first equation means y = —x?, and so 
y’ = x‘. Plugging into the second equation we obtain —x + x* = 0. Factoring we obtain 
x(1 — 2°) = 0. Since we are looking only for real solutions we get either x = 0 or x = 1. 
Solving for the corresponding y using y = —x”, we get two critical points, one being (0,0) 
and the other being (1, —1). Clearly the critical points are isolated. 
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Figure 5.3: The phase portrait with few sample trajectories of 2! = —y—x?, y! =—a+4+y?. 
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Let us compute the Jacobian matrix: 


At the point (0,0) we get the matrix [ o ral and so the two eigenvalues are 1 and —1. As 
the matrix is invertible, the system is almost linear at (0,0). As the eigenvalues are real and 
of opposite signs, we get a saddle point, which is an unstable equilibrium point. Looking at 
the phase portrait, we can see trajectories that would start near (0,0) and end up farther 
away from (0,0). These trajectories may end up at (1,—1), but that is away from (0,0). 

At the point (1,—1) we get the matrix [= = and computing the eigenvalues we get —1, 
—3. The matrix is invertible, and so the system is almost linear at (1,—1). As we have real 
eigenvalues and both negative, the critical point is a sink, and therefore an asymptotically 
stable equilibrium point. That is, if we start with any point (x(0), y(0)) close to (1,—1) as 
an initial condition and plot a trajectory, it approaches (1,—1). In other words, 


lim (x(¢),y(@)) = (1, -D. 

— 00 

As you can see from the diagram, this behavior is true even for some initial points quite far 
from (1,—1), but it is definitely not true for all initial points. | 


Example 5.1.4: Find and analyze the critical points of x’ = y + ye”, y! = x. 


Solution: First let us find the critical points. These are the points where y + ye” = 0 and 
x = 0. Simplifying we get 0 = y + y? = y(y +1). So the critical points are (0,0) and (0, —1), 
and hence are isolated. Let us compute the Jacobian matrix: 


ye 1+ 2Qye* 
1 0 : 


At the point (0,0) we get the matrix [9 }] and so the two eigenvalues are 1 and —1. As 
the matrix is invertible, the system is almost linear at (0,0). And, as the eigenvalues are real 
and of opposite signs, we get a saddle point, which is an unstable equilibrium point. 

At the point (0, —1) we get the matrix [{ ||] whose eigenvalues are 14 4v3. The matrix is 
invertible, and so the system is almost linear at (0,—1). As we have complex eigenvalues with 
positive real part, the critical point is a spiral source, and therefore an unstable equilibrium 
point. 

See Figure 5.4 on the next page for the phase diagram. Notice the two critical points, 
and the behavior of the arrows in the vector field around these points. | 


5.1.5 The trouble with centers 


Recall, a linear system with a center means that trajectories travel in closed elliptical orbits 
in some direction around the critical point. Such a critical point we call a center or a stable 
center. It is not an asymptotically stable critical point, as the trajectories never approach the 
critical point, but at least if you start sufficiently close to the critical point, you stay close to 
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Figure 5.4: The phase portrait with few sample trajectories of x! = y+ ye", y! = x. 


the critical point. The simplest example of such behavior is the linear system with a center. 
Another example is the critical point (0,0) in Example 5.1.1 on page 370. 

The trouble with a center in a nonlinear system is that whether the trajectory goes towards 
or away from the critical point is governed by the sign of the real part of the eigenvalues of 
the Jacobian matrix, and the Jacobian matrix in a nonlinear system changes from point to 
point. Since this real part is zero at the critical point itself, it can have either sign nearby, 
meaning the trajectory could be pulled towards or away from the critical point. 


Example 5.1.5: Find and analyze the critical point(s) of 2’ = y,y’ = -—x+y’. 


Solution: The only critical point is the origin (0,0). The Jacobian matrix is 


0 1 

—1 3y7|° 
At (0,0) the Jacobian matrix is [°, }], which has eigenvalues +7. So the linearization has a 
center. 


Using the quadratic equation, the eigenvalues of the Jacobian matrix at any point (z, y) 
are 


35, .V4-—9y4 

A= 5Y = a 5 : 
At any point where y 4 0 (so at most points near the origin), the eigenvalues have a positive 
real part (y? can never be negative). This positive real part pulls the trajectory away from 
the origin. A sample trajectory for an initial condition near the origin is given in Figure 5.5 
on the next page. _| 


The same process could be carried out with the system x’ = y,y! = —x — y®. This one 
will also have a center as the linearization at the origin, but the non-linear system will have a 
spiral sink at the origin. The moral of the example is that further analysis is needed when 
the linearization has a center. The analysis will in general be more complicated than in the 
example above, and is more likely to involve case-by-case consideration. Such a complication 
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to the vector fields below. Justify. 


Exercise 5.1.3 
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Exercise 5.1.4: Find the critical points and linearizations of the following systems. 
abai=x—y, y=2t+y—l, b) a =—y, yf = 3x + ys”, 


coav=a2ty, yoy te. 


Exercise 5.1.5:* Find the critical points and linearizations of the following systems. 
a) x =sin(ry)+(x¢—-1)*, y’ =y’—-y, b)a’=a+yt+y’, y' =2, 
c) vw =(4@-1)?+y, y=a? ty. 


Exercise 5.1.6: For the following systems, verify they have critical point at (0,0), and find 
the linearization at (0,0). 
alt =aetayte—y, yf =ty-2 beta-y, Y=r-y 


c) v =ar+by+ f(z,y), y' =cx+dy+g(z,y), where f(0,0) = 0, g(0,0) = 0, and all 
first partial derivatives of f and g are also zero at (0,0), that is, 5f (0, 0) = 2£(0, i= 
52 (0,0) = 52(0,0) = 0. 


Exercise 5.1.7: Take the system 2’ = (1 —2)(a+y), y' = (y+3)(a4@ —y). 
a) Find all critical points. 
b) Determine the linearization of this system around each of the critical points. 


c) For each of the critical points, determine the behavior and classify the type of solution 
that the linearized system will have around that critical point. 


Exercise 5.1.8: Take the system x’ = (x? — y)(x +3), y'=(y-l)(x+y+)1). 
a) Find all critical points. 
b) Determine the linearization of this system around each of the critical points. 


c) For each of the critical points, determine the behavior and classify the type of solution 
that the linearized system will have around that critical point. 


Exercise 5.1.9: Take 2’ =(x—y)*, y' =(a+ y)?. 
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a) Find the set of critical points. 
b) Sketch a phase diagram and describe the behavior near the critical point(s). 
c) Find the linearization. Is it helpful in understanding the system? 

Exercise 5.1.10: Take xv! = 27, y' = 2°. 
a) Find the set of critical points. 
b) Sketch a phase diagram and describe the behavior near the critical point(s). 
c) Find the linearization. Is it helpful in understanding the system? 


Exercise 5.1.11:* The idea of critical points and linearization works in higher dimen- 
sions as well. You simply make the Jacobian matrix bigger by adding more functions and 
more variables. For the following system of 3 equations find the critical points and their 
linearizations: 

= gto y =2?-y, y= par. 


Exercise 5.1.12:* Any two-dimensional non-autonomous system x’ = f(z,y,t), yo = 
g(x, y,t) can be written as a three-dimensional autonomous system (three equations). Write 
down this autonomous system using the variables u, v, w. 


Exercise 5.1.13: For the systems below, find and classify the critical points, also indicate 
if the equilibria are stable, asymptotically stable, or unstable. 


a) v =—x4+327,y =-y b)al=a2?+y?-ly=2 
c) w = ye, y =y—at+y? 
Exercise 5.1.14:* For the systems below, find and classify the critical points. 
a)a=—-r4+2*,y'=y b)a =y-y-2,y=-r oc) xe =2y,y =rt+y-1 
Exercise 5.1.15: Find and classify all critical points of the system 


dx dy 
secs iNGe 3 mtd 
di (c+ 1)(a@ —y+3) di 
Exercise 5.1.16: Find and classify all critical points of the system 


= (a — 2)(x —y). 


Faatay Va +ayly-9) 
Exercise 5.1.17: Find and classify the critical point(s) of 2! = —x?, y! = —y?. 
Exercise 5.1.18: Suppose x’ = —xy, y! =x? -—1-y. 
a) Show there are two spiral sinks at (—1,0) and (1,0). 
b) For any initial point of the form (0, yo), find the trajectory. 


c) Can a trajectory starting at (xo, yo) where x9 > 0 spiral into the critical point at 
(—1,0)? Why or why not? 
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Exercise 5.1.19: In the example x' = y, y' = y® — x show that for any trajectory, the 
distance from the origin is an increasing function. Conclude that the origin behaves like is a 
spiral source. Hint: Consider f(t) = (x(t))? + (y(t))” and show it has positive derivative. 


Exercise 5.1.20: Find and analyze all critical points of the system x! = y, y' = —x — y°. 


Use the ideas from Exercise 5.1.19 to show that the solutions to this problem move towards 
the origin as t grows. 


Exercise 5.1.21:* Derive an analogous classification of critical points for equations in one 
dimension, such as x' = f(a) based on the derivative. A point Xo is critical when f (xo) = 0 
and almost linear if in addition f'(x9) # 0. Figure out if the critical point is stable or unstable 
depending on the sign of f'(ao). Explain. Hint: see § 1.7. 
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5.2. Behavior of non-linear systems 


Attribution: [JL], §8.2. 


After this section, you will be able to: 


e Find the trajectories for a non-linear system, 

e Determine if a system is Hamiltonian and use that fact to find the general solution, 
e Use nullclines to help analyze a non-linear system, and 

e Identify basins of attraction and separatrices for a non-linear system. 


5.2.1 Conservative equations 
An equation of the form 
xv" + f(x) =0 


for an arbitrary function f(z) is called a conservative equation. For example the pendulum 
equation is a conservative equation. The equations are conservative as there is no friction 
in the system so the energy in the system is “conserved.” Let us write this equation as a 
system of nonlinear ODE. 


a=y, y =—f(x). 
These types of equations have the advantage that we can solve for their trajectories easily. 
Definition 5.2.1 


Assume that we have an autonomous system of differential equations defining x and y, 


a =f(t,y)  y =9(2,y). 


A trajectory for this system is a curve in the xy-plane that the solution curve (x(t), y(t)) 
will stay on for all t. This curve will generally be given with y as a function of x, or 
the level curve of some function F(x, y). 


For conservative equations, we want to first think of y as a function of x for a moment. 
Then use the chain rule 


where the prime indicates a derivative with respect to t. We obtain ye + f(z) =0. We 
integrate with respect to x to get fy dx + { f(x)dx =C. In other words 


sv + fro dg =. 


We obtained an implicit equation for the trajectories, with different C’ giving different 
trajectories. The value of C’ is conserved on any trajectory. This expression is sometimes 
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called the Hamiltonian or the energy of the system. If you look back to § 1.9, you will notice 
that ye + f(x) =0 is an exact equation, and we just found a potential function. 

Another approach we could use in this case is separable equations, if it works out. The 
idea is that we have the system 


and want to develop a differential equation for y in terms of x. We can write this differential 
equation using some principles from implicit differentiation and parametric equations as 


dy dy /dt 
dx dx /dt’ 
which, for this case, is 
dy _ —f(0) 
de y 
This equation is separable as 
y dy = —f(x) dx 


and we can get to the same implicit equation for the trajectories as before. 


Example 5.2.1: Find the trajectories for the equation x” +2 —2? = 0, which is the equation 
from Example 5.1.1 on page 370. 


Solution: The corresponding first order system is 


/ / 


u=yY, y = —24+ 27. 


Trajectories satisfy 


We solve for y 


2 
y= y/o? + gr +20. 


Plotting these graphs we get exactly the trajectories in Figure 5.1 on page 371. In 
particular we notice that near the origin the trajectories are closed curves: they keep going 
around the origin, never spiraling in or out. Therefore we discovered a way to verify that the 
critical point at (0,0) is a stable center. The critical point at (0,1) is a saddle as we already 
noticed. This example is typical for conservative equations. _| 


Consider an arbitrary conservative equation 2” + f(x) = 0. All critical points occur when 
y = 0 (the z-axis), that is when x’ = 0. The critical points are those points on the z-axis 
where f(x) = 0. The trajectories are given by 


y= 2)-2 f seyde +20 
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So all trajectories are mirrored across the x-axis. In particular, there can be no spiral sources 
nor sinks. The Jacobian matrix is 
0 1 
se) of 


The critical point is almost linear if f’(x) 4 0 at the critical point. Let J denote the Jacobian 
matrix. The eigenvalues of J are solutions to 


0 = det(J — DI) = 2? + f(z). 


Therefore \ = +,/—f’(x). In other words, either we get real eigenvalues of opposite signs 
(if f(z) < 0), or we get purely imaginary eigenvalues (if f’(2) > 0). There are only two 
possibilities for critical points, either an unstable saddle point, or a stable center. There are 
never any sinks or sources. 


5.2.2 Hamiltonian Systems 


A generalization of conservative equations to systems is a Hamiltonian system. This type of 
system has all of the nice properties of conservative equations when converted into systems, 
but allows for more general interactions between x and y. For these systems, the point is 
that the equation has a conserved quantity called a Hamiltonian, which does not change as 
the system evolves in time, which generally represents the energy of the system. Calling this 
function H (a, y), this means that 


d 
—H =U, 
qty) =9 
By the chain rule, this is equivalent to 
OHdx OH dy _ 0 
dx dt Oydt ~ 
One way to satisfy this is with 
dx OH 
dt dy 
5.2 
dy _ alt a 
dt Ox 


and this gives the definition of a Hamiltonian system. 
Definition 5.2.2 


The system 


ae = f(a,y) 


dt (5.3) 


dy 
ce g(x,y) 


is Hamiltonian if there is a function H(z, y) so that f(x,y) = = and g(a, 7) — oH 
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For solving these sorts of systems, we know that 


since that’s how we defined the system. This means that the trajectories of this system are 
given by 

A(z,y)=C 
for a constant C’ determined by initial conditions. So if we can find the function H that 
expresses the system in the form (5.2), then we are done. 


Finding this H is a lot similar to finding solutions to exact equations in § 1.9. First, we 
need to determine if the system is Hamiltonian. Since we want to have that 


OH OH 


we know that 
O°H baal O?H 
anne Bop — 
OxOy ee OxOy 
which shows that 
ist gy =U: 

This is what we can use to check if a system is Hamiltonian; compare to Theorem 1.9.1 for 
exact equations. 

Once we know that a system is Hamiltonian, we can integrate the different components 
of the equation to find the function H. Since f = = then we can write 


A(x,y) = - | few) dy + A(z) 


where A(z) is an unknown function, which can be determined by differentiating this in x and 
setting equal to g(x, y). 


Example 5.2.2: Consider the system of differential equations given by 
v=—4r+3y y' =2r+4y. 


Determine if this system is Hamiltonian and, if so, find the trajectories of the solution. 


Solution: We first check if f; + g, = 0 to see if the system is Hamiltonian. Since f, = —4 
and g, = 4, this means we have a Hamiltonian system. In order to find the function H, we 


use that 
OH _ 


Oy 
Integrating both sides in y gives that 


—f(x,y) = 4x — 3y. 


3 
H(x,y) = day — sy" + A(x) 
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for an unknown function A(x). Differentiating this in x gives 


aH 


which we want to equal 2x + 4y. This gives that A’(x) = 2x so A(x) = x?. Thus, the 
Hamiltonian is given by 


3 
H(g.y9) = x? + Ary — au 


so that the trajectories are defined by 
2 32 
x +4ary — a = . 


for any constant C’. These are sketched in Figure 5.6. 
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Figure 5.6: Vector field and trajectories for a Hamiltonian System 


Note that this system is linear and autonomous. Therefore, we could have solved this 
using those methods as well. For this, we have the coefficient matrix 


—4 3 
a-[ 
and we can find the eigenvalues of this matrix as the roots of 


det(A — AI) = (—4— \)(4— A) — (3) (2) = A? — 22 


whose roots are +\/22 which have opposite signs. Therefore, this will be a saddle point, 
which we see represented in the plot in Figure 5.6. | 
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5.2.3 Separatrices and Basins of Attractions 


If we have an asymptotically stable critical point (29, yo) for an autonomous system of 
differential equation, we know that solutions that start “near” this point will converge to 
it as t > oo. That’s what it means for the point to be asymptotically stable. However, for 
applications, it may be important to know exactly which initial conditions (x(0), y(0)) will 
end up converging to (Xo, yo). This can be particularly relevant when there are multiple 
asymptotically stable equilibrium solutions and we need to determine which one a given 
initial condition will converge to. 


Definition 5.2.3 


Let (20, yo) be an asymptotically stable equilibrium solution for the autonomous system 


> 


¢ ‘=F (x,y). The basin of attraction for (x9, yo) is the set of all points (a,b) where 
the solution to 


@ = F(x,y) 


converges to (2, Yo) as t + oo. 


In general, the basin of attraction for an asymptotically stable critical point is difficult, if 
not impossible, to find analytically. The main approach here is to use a direction field to 
approximate the basin of attraction. 


Example 5.2.3: Find all asymptotically stable critical points for the autonomous system 


dx dy 
ap 7 alt — 2x — dy) a 7 YT AG — 3a — 2y) 


and determine an approximate basin of attraction for each. 


Solution: We want to start by finding the critical points for this system, classifying them to 
determine if they are asymptotically stable, and then use a slope field to try to find the basin 
of attraction. In order to have a critical point, we need to have both a and oy equal to zero. 
This means that we need 


[c=Oor7—2e2-—5y=0] and [y=-—lor5—3r-2y=0). 


This results in the points (0,—1), (0,5/2), (6, -1), and (1,1). In order to classify each of these 
critical points, we need to find the Jacobian matrix for this system, which is 


= | Pe 2p)... |f = 42 =—oy —52x 
He =|@ a eae 2 ap 


and then we want to plug each critical point into this matrix in turn. 


Plugging in (0,—1) gives [29], which has eigenvalues of 12 and 7, and so is a nodal 
source, which is unstable. Plugging in (0, 5/2) gives Ee: Lae which has eigenvalues of 
—11/2 and —7, which is a nodal sink, and so asymptotically stable. Plugging in (6, —1) gives 


les a |; which has eigenvalues at —11 and —12, giving an asymptotically stable nodal sink. 
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The last point at (1,1) gives be Tk which is not triangular, and so does not have easily 
identifiable eigenvalues. We could use trace-determinant analysis to classify (§ 4.7) or we can 
just compute the eigenvalues. Those are found by the roots of 


det(A — AZ) = (—2 — 4)(—4 — A) — (-5)(-2) = A? +.6A-2 


and since the last term is negative, we know we are going to get roots of opposite signs, so 
this is an unstable saddle point. The actual eigenvalues are 


6+ V36-4D-2)_ og VE ng 
2 2 | 


So, this means we have two asymptotically stable critical points, (0,5/2) and (6,—1). We 
need to look at a slope field to determine the approximate basin of attraction for each of 
these points. 
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Figure 5.7: Plots showing the slope field for Example 5.2.3 (left) and a slope field combined with 
approximate basins of attraction (right). 


From Figure 5.7, we can see that there is a sort of dividing line between the two nodal 
sinks. If the solution starts on one side of the line, it funnels into one critical point, and on 
the other side, it heads to the other one. This dividing line also seems to pass through the 
saddle point at (1,1), which is not a coincidence, as we will see later. | 


Another interesting feature of these regions is the boundary of them. This is a curve that 
separates solutions that converge to the asymptotically stable equilibrium solution and those 
that don’t. This leads to another definition. 


390 


Definition 5.2.4 


=> 
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Consider the autonomous system z” = F(x, y). A separatrix (plural separatrices) is a 


curve in the plane that separates trajectories that have different long-term behaviors 
ol solutions to 2 = F(a.y)e 


The boundary of a basin of attraction is a separatrix because the long-term behavior 
inside the curve (converging to the asymptotically stable critical point) is different from the 
behavior outside the curve (going somewhere else). These dividing curves also show up in 


other contexts. 


Example 5.2.4: Analyze the system 
da 
dt 
dy 
dt 
in the context of separatrices. 


—2% 


Solution: This is a linear, homogeneous system, so we can analyze it via that approach. For 


-1-1 


the coefficient matrix A = [ =} %' |, we have eigenvalues as the roots of (—1 — A)(—A) — 2, or 
d? + 4 —2. Therefore, the eigenvalues here are 1 and —2. For 1 we have eigenvector [ ',] and 
for —2, an eigenvector is [+]. We can see what this looks like on a slope field in Figure 5.8. 


There are no asymptotically stable crit- 
ical points here, so there are no basins of 
attraction. However, there are two distinct 
behaviors of the solution curve. It is going 
away from the origin, but it could go to the 
top left, or to the bottom right. Both of 
those make sense based on the slope field 
here. So how do we know which way it goes? 
The line drawn on the right side of Figure 5.8 
seems to divide these two regions up. If the 
solution starts above the line, it goes to the 
top left, otherwise, it goes to the bottom 
right. This is the separatrix for this saddle 
point. 

But what is that line? If we inspect the 
graph more closely, the separatrix here is the 
straight-line solution that converges to zero 
over time; the one particular solution that 
does not go off to infinity because it only 
has the e~~ term in it. So the straight-line 
solutions that flow into saddle points divide 
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Figure 5.8: Plot showing the slope field for 
Example 5.2.4 with a sketch of the separatriz. 


what happens on the two sides of it. This is a very common fact in looking at separatrices: if 
they go through a critical point, they generally do so as the in-flowing solution from a saddle 
point. All of the examples we have seen so far with separatrices have done exactly this. __ | 
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5.2.4 Nullclines 


When trying to find critical points for a non-linear, autonomous system, we need all (both, 
in the case of two component systems) of the equations to be zero. What happens if only 
one of the equations is zero? This is a lot easier to find, and can also give us a fair bit of 
information. 


Definition 5.2.5 


Consider the autonomous two-component system 


dx dy 


A nullcline for this system is a curve where either f(x,y) =0 or g(x,y) = 0. We can 


also be more specific and use the term z-nullcline for the curve(s) where @ = 0 and 


d 
y-nullcline for where 4 = 0. 


The way we can use these nullclines is to know in general which direction the solution 
curve will move in different regions of the plane. Assuming that all of the functions involved 
are continuous, if we know that the solution at a given point will move to the right, that is, 
if de > 0, then we know that the solution will continue to move to the right until we cross an 
x-nullcline. If the solution starts going back to the left, this means that de becomes negative, 
and so must cross zero, which is where a nullcline is. 

In addition, we know that along an zx-nullcline, i = 0, so the solution can only be moving 
in the y direction, that is, vertically. If we can determine in which direction the solution 
graph will cross the nullclines, this can also be helpful and useful. It doesn’t give as much 
information as a full slope field or trajectory plot, but it can give a general idea of what is 
going to happen to the solution over time. 


Example 5.2.5: Use a nullcline analysis to determine the overall behavior of solutions to 
the system 

dx 7 

S=-De-y A= +3je-2). 
Solution: We can get the equations for nullclines from the factors of each of the differential 
equations here. For z-nullclines, we get y = 1 and y = z, and for y-nullclines, we get 
x = —3 and x = 2. Once we have these lines, we need to determine what happens within 
each of the regions on the resulting graph. For example, if we look in the region above 
y = 1, above y = z and right of x = 2, we can plug in, for example (3,4). At this point 
 — (4—1)(3 —4) =-3 < 0 and a = (3 + 3)(3 — 2) =6 > 0. Therefore, the solution here 
moves up and to the left. We can fill in all of the other regions in a similar manner. This is 
shown on the left of Figure 5.9. 

Based on the nullcline diagram here, we can see that there seems to be some sort of 
spiraling behavior around both (2,2) and (—3, —3), which we know are critical points because 
the two different nullclines intersect there. From this alone, we can’t really tell if they are 
sources, sinks, or centers, but we do get a general idea of the behavior. We also see what looks 
like saddles at (—3, 1) and (2,1), since these critical points have two opposite arrows pointing 
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Figure 5.9: Plots showing the nullcline diagram for Example 5.2.5 (left) and a slope field for the 
same differential equation (right). 


towards this point (corresponding to the negative eigenvalue of the linearized system), and 
two opposite arrows pointing away from the point (corresponding to the positive eigenvalue. 
The slope field seems to validate all of these general discussions from the nullcline diagram| 


Example 5.2.6: Use a nullcline analysis to determine the overall behavior of solutions to 


the system 
dx dy 


gato DYtY) Gg =—2X(e+y). 
Solution: As with the previous example, we can find the nullcline equations from the factors 
above. The z-nullclines are at x = 3 and y = —1, and the y-nullclines are at y = 2 and 
x = —y. We can plug in points to fill in the nullcline diagram like before, and compare to 
the slope field, shown in Figure 5.10. 

In this diagram, we see some of the other types of critical points and what they look like 
through nullclines. The point at (3,2) looks like a nodal source, because all of the arrows 
point away from that point, and (3, —3) looks like a nodal source. Finally, we see a potential 
saddle at (1,—1) because of the patterns of the arrows, all of which are also shown in the 
slope field for this system. 

An extra point with this type of result is that we know we can not cross the nullclines at 
x =3 and y = 2. For instance, the line x = 3 is an x-nullcline. This means that the solution 
must cross the line moving vertically. However, it is a vertical line, and there is no way to 
cross a vertical line moving vertically. The same argument applies to y = 2. We can also see 
this by the fact that the arrows on either side of the line both point into or away from these 
nullclines. _| 


5.2.5 Exercises 


Exercise 5.2.1: Find the implicit equations of the trajectories of the following conservative 
systems. Next find their critical points (if any) and classify them. 
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Figure 5.10: Plots showing the nullcline diagram for Example 5.2.6 (left) and a slope field for 
the same differential equation (right). 


a) xv” +24+2°2=0 b) 6” + siné@ = 0 
c) 2” +(z-1)(2+1) =0 d) a" +27+1=0 


Exercise 5.2.2:* Find the implicit equations of the trajectories of the following conservative 
systems. Next find their critical points (if any) and classify them. 


a)a"+a7=4 b) x” +e? =0 c) a + (x+1)e7 =0 


Exercise 5.2.3:* The conservative system x" + x? = 0 is not almost linear. Classify its 
critical point(s) nonetheless. 


Exercise 5.2.4: Determine if the following system is Hamiltonian. If it is, find the general 
solution in the form H(x,y) = C and sketch some of the trajectories. 


dz _ 9 dy 
dt et 


= oe = 


Exercise 5.2.5: Determine if the following system is Hamiltonian. If it is, find the general 
solution in the form H(x,y) = C and sketch some of the trajectories. 


dx 


dy 
—SAp—JytF =—=—5 —1. 
di £ y+ di u+y 


Exercise 5.2.6: Determine if the following system is Hamiltonian. If it is, find the general 
solution in the form H(x,y) = C and sketch some of the trajectories. 


dx 


d 
ae 7 ey + 3y" pa — ey +e. 
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Exercise 5.2.7: Determine if the following system is Hamiltonian. If it is, find the general 
solution in the form H(x,y) = C and sketch some of the trajectories. 


Exercise 5.2.8: Consider a generic thing on a spring, with displacement u and velocity v. 
Assume that 
mu” +ku=0, 


where m and k are some positive constants. 


a) Rewrite this equation as a first-order system in u and v. 

b) Find a Hamiltonian function for this system (in terms of u and v). 
c) What shapes are the level curves of the Hamiltonian function? 

d) Does this system have a basin of attraction? Explain briefly. 


Exercise 5.2.9: Suppose f is always positive. Find the trajectories of x" + f(x’) = 0. Are 
there any critical points? 


Exercise 5.2.10: Suppose that x’ = f(x,y), y’ = g(x,y). Suppose that g(x,y) > 1 for all x 
and y. Are there any critical points? What can we say about the trajectories at t goes to 
infinity? 


d d 
Exercise 5.2.11: Here is the direction field for the system | = y* — 3y, 7 


The critical points are (0,0), (4,0), (0,3), and (4,3). Draw the nullclines on the plot. What 
do the nullclines tell us about the critical points? 


= 77 — Ag. 
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Exercise 5.2.12: Nullclines apply to linear systems as well, although since we can often 


solve those explicitly they’re less necessary. Construct the nullcline diagram for the system 


d 
= =—3r+y, a = 6x + 2y, and use it to classify (by type) the equilibrium point at the 
origin. What is the linearization of this system at (0,0)? 


d d 
Exercise 5.2.13: Consider the system ; = =29 > y; a = -y+ 2”. 


a) Find all equilibrium solutions. 


b) Sketch all nullclines for this system on a single diagram. Label each region, and use 
these results to classify each equilibrium point. 


Exercise 5.2.14: Nullclines need not be lines. Consider the system 


a) Find all critical points of this system. 


b) Sketch the nullcline diagram and label all regions DL, DR, UL, or UR. Classify (according 
to type) any critical point(s) that can be classified using this analysis. 


c) Two critical points cannot be classified using the nullcline analysis. Classify these 
(again according to type) using the Jacobian. 


Exercise 5.2.15: Consider the system 


@or—-y 42 


5.4 
ee (5.4) 
dt 


22 — y? 
a) Find all critical points of (5.4). 


b) Create the nullcline diagram for the system, labelling each region as one of UL, UR, 
DL, or DR. Use this information to classify two critical points according to type. 


c) Use the Jacobian matrix to classify any remaining critical points. 


d) Is there a conserved quantity (Hamiltonian function) for this system? If so, find one. If 
not, explain why not. 


d. 
Exercise 5.2.16: For a conflict between two armies, Lanchester’s Law asserts that | = —-ay 


d 
and ane —6x, where x and y are the two populations, and a and { are some positive 


constants. 
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a) Find a Hamiltonian function for this system satisfying H(0,0) = 0. 
b) Classify the critical point at the origin according to type and stability. 


c) Assume that we are just looking at the first quadrant, since the populations are non- 
negative. Find the curve along which the Hamiltonian function is zero, and explain its 
significance in terms of who wins the conflict. 


Exercise 5.2.17: Consider the non-linear system 


d 
77 = 4¢ — By — 2(z? +7), 


dy _ 
dt 


32 + dy — y(a? + y’). (5.5) 
a) (5.5) has a critical point at the origin. What is the linearization of (5.5) at the origin? 
b) Demonstrate that (5.5) is locally linear in a neighborhood of the origin. 

c) Classify the origin according to its type and stability. 


Exercise 5.2.18: Consider the system of differential equations 


die 
dt 
which has slope field sketched below. 


(?-ly S=(y-3)\(y-l1)¢ (5.6) 
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a) Find and classify all critical points of the system (5.6). 
b) Draw any separatrices that you can spot on the slope field. 


c) Do any of these critical points have a basin of attraction? If so, sketch out what regions 
of the plane correspond to a basin of attraction for those critical points. 
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Exercise 5.2.19: Consider the system of differential equations 


& = (2-wNy + Ye +1) Y (e+ 2)(@ —1)y (5.7) 


which has slope field sketched below. 
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a) Find and classify all critical points of the system (5.7). 
b) Draw any separatrices that you can spot on the slope field. 


c) Do any of these critical points have a basin of attraction? If so, sketch out what regions 
of the plane correspond to a basin of attraction for those critical points. 
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5.3. Applications of nonlinear systems 


Attribution: [JL], §8.3. 


After this section, you will be able to: 


e Use non-linear systems to model the motion of a pendulum, and 
e Use non-linear systems to model population dynamics like predator-prey and 
competing species models. 


In this section we study two very standard examples of nonlinear systems. First, we look 
at the nonlinear pendulum equation. We saw the pendulum equation’s linearization before, 
but we noted it was only valid for small angles and short times. Now we find out what 
happens for large angles. Next, we look at the predator-prey equation, which finds various 
applications in modeling problems in biology, chemistry, economics, and elsewhere. 


5.3.1 Pendulum 


The first example we study is the pendulum equation 6” + #sin@ = 0. Here, @ is the angular 
displacement, g is the gravitational acceleration, and L is the length of the pendulum. In 
this equation we disregard friction, so we are talking about an idealized pendulum. 

This equation is a conservative equation, so we can use our analysis 
of conservative equations from the previous section. Let us change the 
equation to a two-dimensional system in variables (0, w) by introducing ry 
the new variable w: rf) 


fT = L-gind , 


The critical points of this system are when w = 0 and —#sin@ = 0, or in other words if 
sin 0 = 0. So the critical points are when w = 0 and @ is a multiple of 7. That is, the points are 
...(—27, 0), (—7, 0), (0,0), (7, 0), (27,0) .... While there are infinitely many critical points, 
they are all isolated. Let us compute the Jacobian matrix: 


we) (4) 


0 | 
3(—fsind) 2.(—$siné) =| 0os6 a 


00 


For conservative equations, there are two types of critical points. Either stable centers, or 
saddle points. The eigenvalues of the Jacobian matrix are \ = +,/—4 cos. 

The eigenvalues are going to be real when cos@ < 0. This happens at the odd mul- 
tiples of 7. The eigenvalues are going to be purely imaginary when cos@ > 0. This 
happens at the even multiples of 7. Therefore the system has a stable center at the points 


...(—27, 0), (0,0), (27,0) ..., and it has an unstable saddle at the points .. . (—37,,0), (—7, 0), (7, 0), (37, 0)... 


Look at the phase diagram in Figure 5.11 on the facing page, where for simplicity we let 
| 
; : 
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Figure 5.11: Phase plane diagram and some trajectories of the nonlinear pendulum equation. 


Since this is a pendulum without friction, we can characterize the two differnet types 
of trajectories here. There are the curves running along the top and bottom of the phase 
portrait that look somewhat like sine waves. These graphs never cross the x-axis, which is 
the line w = 0. Therefore, these are trajectories where the pendulum never stops moving; it 
just keeps spinning around in full circles forever, crossing through all possible values of 6. 
The other type of trajectory are the ellipses around each of the stable equilibrium solutions. 
In these cases, the graph only spans a specific range of 6 values, represented by the reduced 
x range of the ellipse, and cycles there forever. This represents a pendulum that does not 
have enough energy to make a full circle, and just oscillates back-and-forth to a fixed height 
forever. 

In the linearized equation we have only a single critical point, the center at (0,0). Now 
we see more clearly what we meant when we said the linearization is good for small angles. 
The horizontal axis is the deflection angle. ‘The vertical axis is the angular velocity of the 
pendulum. Suppose we start at 9 = 0 (no deflection), and we start with a small angular 
velocity w. Then the trajectory keeps going around the critical point (0,0) in an approximate 
circle. This corresponds to short swings of the pendulum back and forth. When @ stays small, 
the trajectories really look like circles and hence are very close to our linearization. 

When we give the pendulum a big enough push, it goes across the top and keeps spinning 
about its axis. This behavior corresponds to the wavy curves that do not cross the horizontal 
axis in the phase diagram. Let us suppose we look at the top curves, when the angular 
velocity w is large and positive. Then the pendulum is going around and around its axis. 
The velocity is going to be large when the pendulum is near the bottom, and the velocity is 
the smallest when the pendulum is close to the top of its loop. 

At each critical point, there is an equilibrium solution. Consider the solution 6 = 0; 
the pendulum is not moving and is hanging straight down. This is a stable place for the 
pendulum to be, hence this is a stable equilibrium. 

The other type of equilibrium solution is at the unstable point, for example 6 = 7. Here 
the pendulum is upside down. Sure you can balance the pendulum this way and it will stay, 
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but this is an unstable equilibrium. Even the tiniest push will make the pendulum start 
swinging wildly. 

See Figure 5.12 for a diagram. The first picture is the stable equilibrium # = 0. The 
second picture corresponds to those “almost circles” in the phase diagram around @ = 0 when 
the angular velocity is small. The next picture is the unstable equilibrium 6 = 7. The last 
picture corresponds to the wavy lines for large angular velocities. 


We ee 


g=0 Small angular velocities d=" Large angular velocities 


Figure 5.12: Various possibilities for the motion of the pendulum. 


The quantity 
1 
5 — + cos 0 

is conserved by any solution. This is the energy or the Hamiltonian of the system. 


We have a conservative equation and so (exercise) the trajectories are given by 


2 
w = +£4/—* cosd +O, 


for various values of C’. Let us look at the initial condition of (09,0), that is, we take the 
pendulum to angle 0, and just let it go (initial angular velocity 0). We plug the initial 
conditions into the above and solve for C’ to obtain 


2 
C= aA COS Oo. 


Thus the expression for the trajectory is 


2 
Ww = +1/= v/cosd — cos Op. 


Let us figure out the period. That is, the time it takes for the pendulum to swing back 
and forth. We notice that the trajectory about the origin in the phase plane is symmetric 
about both the 6 and the w-axis. That is, in terms of 9, the time it takes from 09 to —9p is 
the same as it takes from —@) back to 69. Furthermore, the time it takes from —6p to 0 is 
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the same as to go from 0 to 4. Therefore, let us find how long it takes for the pendulum to 
go from angle 0 to angle 09, which is a quarter of the full oscillation and then multiply by 4. 
We figure out this time by finding a and integrating from 0 to @. The period is four 
times this integral. Let us stay in the region where w is positive. Since w = a inverting we 
get 
dt L 1 
dd \l 2g \/cos 8 — cos Oo 


Therefore the period T is given by 


00 
THA = | : d6. 
29g Jo /cosd — cos % 


The integral is an improper integral, and we cannot in general evaluate it symbolically. We 
must resort to numerical approximation if we want to compute a particular 7’. 
Recall from § 2.4, the linearized equation 6” + #6 = 0 has period 


L 
(ines = 274/—. 
g 


T—Tiinear 
T 


We plot 7, Tiinear, and the relative error in Figure 5.13. The relative error says how 
far is our approximation from the real period percentage-wise. Note that Tiincar is simply a 
constant, it does not change with the initial angle 0). The actual period T gets larger and 
larger as 0) gets larger. Notice how the relative error is small when 6p is small. It is still only 
15% when 6) = 5, that is, a 90 degree angle. The error is 3.8% when starting at 7, a 45 
degree angle. At a 5 degree initial angle, the error is only 0.048%. 
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Figure 5.13: The plot of T and Tinear with £ = 1 (left), and the plot of the relative error 
L 


Poincar (right), for 09 between 0 and 7/2. 


While it is not immediately obvious from the formula, it is true that 


lim T' = oo. 


O0tn 
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That is, the period goes to infinity as the initial angle approaches the unstable equilibrium 
point. So if we put the pendulum almost upside down it may take a very long time before 
it gets down. This is consistent with the limiting behavior, where the exactly upside down 
pendulum never makes an oscillation, so we could think of that as infinite period. 


5.3.2 Predator-prey or Lotka—Volterra systems 


One of the most common simple applications of nonlinear systems are the so-called predator- 
prey or Lotka—Volterra* systems. For example, these systems arise when two species interact, 
one as the prey and one as the predator. It is then no surprise that the equations also see 
applications in economics. The system also arises in chemical reactions. In biology, this 
system of equations explains the natural periodic variations of populations of different species 
in nature. Before the application of differential equations, these periodic variations in the 
population baffled biologists. 

We keep with the classical example of hares and foxes in a forest, it is the easiest to 
understand. 

x = # of hares (the prey), 


y = # of foxes (the predator). 


When there are a lot of hares, there is plenty of food for the foxes, so the fox population 
grows. However, when the fox population grows, the foxes eat more hares, so when there are 
lots of foxes, the hare population should go down, and vice versa. The Lotka—Volterra model 
proposes that this behavior is described by the system of equations 


i = (a = by)x, 
y = (ca ~~ d)y, 


where a, b,c, d are some parameters that describe the interaction of the foxes and hares’. In 
this model, these are all positive numbers. 

Let us analyze the idea behind this model. The model is a slightly more complicated idea 
based on the exponential population model. First expand, 


x’ = (a — by)x = ax — bya. 

The hares are expected to simply grow exponentially in the absence of foxes, that is where the 
ax term comes in, the growth in population is proportional to the population itself. We are 
assuming the hares always find enough food and have enough space to reproduce. However, 
there is another component —byx, that is, the population also is decreasing proportionally to 
the number of foxes. Together we can write the equation as (a — by)z, so it is like exponential 
growth or decay but the constant depends on the number of foxes. 

The equation for foxes is very similar, expand again 


y = (cx — d)y = cry — dy. 


*Named for the American mathematician, chemist, and statistician Alfred James Lotka (1880-1949) and 
the Italian mathematician and physicist Vito Volterra (1860-1940). 
‘This interaction does not end well for the hare. 
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The foxes need food (hares) to reproduce: the more food, the bigger the rate of growth, hence 
the cry term. On the other hand, there are natural deaths in the fox population, and hence 
the —dy term. 

Without further delay, let us start with an explicit example. Suppose the equations are 


x’ = (0.4—0.01ly)z, y = (0.003x — 0.3)y. 


See Figure 5.14 for the phase portrait. In this example it makes sense to also plot x and y as 
graphs with respect to time. Therefore the second graph in Figure 5.14 is the graph of x and 
y on the vertical axis (the prey x is the thinner blue line with taller peaks), against time on 
the horizontal axis. The particular solution graphed was with initial conditions of 20 foxes 
and 50 hares. 
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Figure 5.14: The phase portrait (left) and graphs of x and y for a sample solution (right). 


Let us analyze what we see on the graphs. We work in the general setting rather than 
putting in specific numbers. We start with finding the critical points. Set (a — by)x = 0, 
and (cx — d)y = 0. The first equation is satisfied if either x = 0 or y = 2/s. If x = 0, the 
second equation implies y = 0. If y = 2/b, the second equation implies x = 4/c. There are two 
equilibria: at (0,0) when there are no animals at all, and at (4/c,2/b). In our specific example 
x = 4/- = 100, and y = */s = 40. This is the point where there are 100 hares and 40 foxes. 

We compute the Jacobian matrix: 


a—by —bx 

cy cx Al : 
At the origin (0,0) we get the matrix [¢ °,], so the eigenvalues are a and —d, hence real and 
of opposite signs. So the critical point at the origin is a saddle. This makes sense. If you 
started with some foxes but no hares, then the foxes would go extinct, that is, you would 
approach the origin. If you started with no foxes and a few hares, then the hares would keep 
multiplying without check, and so you would go away from the origin. 
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OK, how about the other critical point at (4c, */b). Here the Jacobian matrix becomes 


The eigenvalues satisfy \? + ad = 0. In other words, \ = +iVad. The eigenvalues being 
purely imaginary, we are in the case where we cannot quite decide using only linearization. 
We could have a stable center, spiral sink, or a spiral source. That is, the equilibrium could 
be asymptotically stable, stable, or unstable. Of course I gave you a picture above that seems 
to imply it is a stable center. But never trust a picture only. Perhaps the oscillations are 
getting larger and larger, but only very slowly. Of course this would be bad as it would imply 
something will go wrong with our population sooner or later. And I only graphed a very 
specific example with very specific trajectories. 

How can we be sure we are in the stable situation? As we said before, in the case of 
purely imaginary eigenvalues, we have to do a bit more work. The main approach that can 
be used here is to directly solve for the trajectories. We can determine a differential equation 
that relates x to y by writing 


dy dy/dt — (cx—d)y 
dx dx/dt (a—by)x 


This is a separable first order equation, which we can rewrite as 


=f = 
a ec d 
y a 


da. 


After simplifying the fractions, we can integrate this to obtain the implicit solution 
aln(y) — by = cz —dln(x) + C 


or 


C =aln(y) + dln(x) — cx — by. (5.8) 


Since we ended up finding a trajectory here that sketches out a closed curve, and we know 
that our solutions must lie on these trajectories, that tells us that, for a fact, we do have 
closed loops here, and the critical point is stable. 

However, we can go a bit farther than this with our discussion here. If we let D = e© in 
(5.8), we can rearrange the expression to get that 


and 
_ ar = yrate ey 
and based on how our trajectory setup works, we know that this D will be conserved along 
the flow of the solution. That is, if the initial condition has a specific value of D, the solution 
will continue to have that same value for all t. This idea came up before in the idea of 
conservative or Hamiltonian systems in § 5.2. Such a quantity is called the constant of motion, 
and this forces the trajectory to go in closed loops. Let us check D really is a constant of 
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motion. How do we check, you say? Well, a constant is something that does not change with 
time, so let us compute the derivative with respect to time: 


D = = ay” lola d ca by a y dar d— Ly! eee by a yr e —cx— by — cr’ _ by’). 
Our equations give us what x’ and 4’ are so let us plug those in: 


pD' =f ay* "(cx oe d)yxte fy an yd (a _ by)xe~ oy 
+ y*r%e—@—™ (—c(a — by)x — b(cx — d)y) 
ype (a(cx — d) + d(a— by) + (—c(a — by)x — b(cx — dy)) 


So along the trajectories D is constant. In fact, the expression D = eer gives us an implicit 
equation for the trajectories. In any case, once we have found this constant of motion, it 
must be true that the trajectories are simple curves, that is, the level curves of —4 a It 
turns out, the critical point at (4/c, 4/b) is a maximum for D (left as an exercise). So (4/c, ¢/b) 
is a stable equilibrium point, and we do not have to worry about the foxes and hares going 
extinct or their populations exploding. 

One blemish on this wonderful model is that the number of foxes and hares are discrete 
quantities and we are modeling with continuous variables. Our model has no problem with 
there being 0.1 fox in the forest for example, while in reality that makes no sense. The 
approximation is a reasonable one as long as the number of foxes and hares are large, but it 
does not make much sense for small numbers. One must be careful in interpreting any results 
from such a model. 

An interesting consequence (perhaps counterintuitive) of this model is that adding animals 
to the forest might lead to extinction, because the variations will get too big, and one of 
the populations will get close to zero. For example, suppose there are 20 foxes and 50 hares 
as before, but now we bring in more foxes, bringing their number to 200. If we run the 
computation, we find the number of hares will plummet to just slightly more than 1 hare in 
the whole forest. In reality that most likely means the hares die out, and then the foxes will 
die out as well as they will have nothing to eat. 


Example 5.3.1: Consider the system 


=(24y-6)e yi =(2—z)y. 


This fits the description of a predator-prey model. Which species is the predator? Find 
and analyze the critical points of this system, and draw a sketch of the phase portrait, with 
arrows to indicate the direction of flow around this portrait. 


Solution: If we expand out the equations in the model, we get 
eg =2Qey-—6c yf = 2Qy—xYy. 


These equations show that, if y = 0, « would decay away in time, and if x = 0, y would grow 
indefinitely. This means that x is the predator and y is the prey in this relationship. For 
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critical points, we can look back at the factored version of the equations to see that we get 
one critical point at (0,0) and one critical point at (2,3). Since this is a predator-prey model, 
we know that we will have cycles around the critical point at (2,3). 

The direction of these cycles is determined by the predator-prey relationship. If we start 
with «x large (greater than 2) and y small (less than 3), then there are a lot of predators 
and few prey. This implies that the next thing to happen is that the predator population 
will decrease because there is not enough prey. We can also see this from the equations; if 
x >2andy <3, then both de and a will be negative. Similarly, if y is large and «x is small, 
there are a lot of prey and few predators, so the prey population will continue to grow, while 
the predators also grow because of the excess of food. This means that the populations will 
follow these trajectories in a clockwise direction. 

For the actual trajectories, we can solve for them in the same way as the calculations 
before this example. We can rewrite this system to give a differential equation for the 


trajectories as 
dy _ (2-a)y 


dx (2y—6)a 


which can be rearranged as a separable equation to 


(Sa (2)u 


2y — 6In(y) + C = 2In(x) —& 


Solving this gives 


or 
C = 2In(x) + 6In(y) — # — 2y. 


This will be used to draw the trajectories in Figure 5.15. 

This can also be seen using a nullcline analysis. The nullclines we need to draw are x = 0, 
y =0, x =2 and y = 3. Our discussion previously shows that the arrow in the bottom-right 
quadrant should point to the lower left, and the arrow in the top left should point up and 
right. We can fill in the other two quadrants to see that the solution should move around the 
circle in a clockwise direction. Figure 5.15 shows the nullcline image and trajectory curves 
for this example. | 


5.3.3 Competing Species systems 


Another application of non-linear systems that also works with population models is a 
competing species interaction. The setup is that there are two species that live in the same 
environment, and need to compete over resources. This means that both species will grow on 
their own, but when the two species interact, it is negative for both species. This gives rise 
to a system of differential equations of the form 


dx 
dt 


dy 
Fs cy — dxy 


= ax — bry 
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Figure 5.15: The nullcline analysis image (left) and graphs of several sample trajectories with 
arrows (right) for Example 5.3.1 


if both populations grow exponentially, or 


dx 

a ax(K — x) — bry 
dy 

en AE ang) 

HE CULM — y) — dey 


if both species grow logistically. The numbers here are all positive constants that explain 
how the different populations affect growth rates. For the logistic model, let’s look at the 
equilibrium solutions. For this, we need 


x(ak — ax — by) =0 y(cM — cy — dr) =0 


which gives equilibrium solutions at (0,0), (0,/), and (K,0), all of which result in one (or 
both) of the species being extinct. The other equilibrium solution is more interesting, because 
it involves both species coexisting. This happens when 


by = aK — ax cy =cM — dz. 


Solving this gives a critical point with x > 0 and y > 0. 
The Jacobian matrix for this system is 


ak — 2ax — by —bx 
J(x@,y) = —dy cM — 2cy — dx} ° 

Unlike the predator-prey system that always had the same type of equilibriums solution 
every time, there are multiple options for how this system can behave based on the values of 
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a, b, c, d, K, and M. It is possible that the coexistence equilibrium solution will be a nodal 
sink, so that all nearby solutions will converge to it over time, and the species will continue 
to exist in harmony. However, it is also possible that the coexistence solution is a saddle and 
the solutions at (AK,0) and (0, /) are sinks. This means that coexistence is unstable, and 
that over time, the populations will converge to one of the other two equilibrium solutions, 
meaning that one of the species will die out as time goes on. Determining which will survive 
will require a numerical model since these equations can not be solved analytically. 


Example 5.3.2: Analyze the competing species model given by the system of differential 
equations 
xv =2(4—2—-2y) y =y(7—-y— 32). 


Is the coexistence solution stable or unstable? What will happen to the populations over 
time? 


Solution: Solving for the equilibrium solutions gives (0,0), (4,0), (0,7), and the coexistence 
solution where 
4—g=2y y=7-—3e. 


Simplifying this gives 
4—xz=14-62 


or x = 2. The second equation then implies that y = 1. 
The Jacobian for this system is 


_ |4— 2x — 2y —2% 
I(t.) = —3y 7 —2y—- 7 , 


Evaluating this matrix at the point (2,1) gives 


—2 -4 

—3 -1]’ 
which we need to find the eigenvalues to classify what type of linearized solution we have 
here. These are determined by 


(—2 —A)(-1—A) —-12 = 74+ 3A-9=0. 
Thus, the eigenvalues are given by 


_ -3+ V9+36 


‘ 2 


which will be real with opposite signs. Therefore, this equilibrium solution is a saddle, and 
unstable. To confirm this, we can also check the equilibrium solutions at (4,0) and (0,7). 
For (4,0), we get the matrix 

—8 —8 

oe 
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which is a nodal sink. For (0,7), we get 
—10 0O 
—21 -—7 
which is also a nodal sink. Thus, we see that the coexistence equilibrium solution is unstable, 


and both of the equilibrium solutions with one species extinct are stable. Therefore, over 
time, one of the two species will die off depending on the initial population. al 


Showing that a system of equations has a stable solution can be a very difficult problem. 
When Isaac Newton put forth his laws of planetary motions, he proved that a single planet 
orbiting a single sun is a stable system. But any solar system with more than 1 planet 
proved very difficult indeed. In fact, such a system behaves chaotically (see § 5.5), meaning 
small changes in initial conditions lead to very different long-term outcomes. From numerical 
experimentation and measurements, we know the earth will not fly out into the empty space 
or crash into the sun, for at least some millions of years or so. But we do not know what 
happens beyond that. 


5.3.4 Exercises 


Exercise 5.3.1: Take the damped nonlinear pendulum equation 6” + 0’ + (9/L) sin@ = 0 
for some 41 > 0 (that is, there is some friction). 


a) Suppose ps = 1 and 9/1 = 1 for simplicity, find and classify the critical points. 


b) Do the same for any « > 0 and any g and L, but such that the damping is small, in 
particular, pi? < 4(9/L). 


c) Explain what your findings mean, and if it agrees with what you expect in reality. 


Exercise 5.3.2:* Take the damped nonlinear pendulum equation 6” + 6’ +(9/L) sin@ = 0 for 
some 1 > 0 (that is, there is friction). Suppose the friction is large, in particular ? > 4(9/L). 


a) Find and classify the critical points. 
b) Explain what your findings mean, and if it agrees with what you expect in reality. 


Exercise 5.3.3: Suppose the hares do not grow exponentially, but logistically. In particular 
consider 
x’ = (0.4—0.0ly)x — yz’, y’ = (0.0032 — 0.3)y. 


For the following two values of y, find and classify all the critical points in the positive 
quadrant, that is, for x > 0 and y > 0. Then sketch the phase diagram. Discuss the 
implication for the long term behavior of the population. 


a) y= 0.001, b) += 0.01. 


Exercise 5.3.4:* Suppose we have the system predator-prey system where the foxes are also 
killed at a constant rate h (h foxes killed per unit time): x’ = (a — by)az, y’ = (ca — d)y —h. 
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a) Find the critical points and the Jacobian matrices of the system. 


b) Put in the constants a = 0.4, b = 0.01, c = 0.003, d= 0.3, h = 10. Analyze the critical 
points. What do you think it says about the forest? 


Exercise 5.3.5 (challenging):* Suppose the foxes never die. That is, we have the system 
x’ = (a— by)a, y! = cry. Find the critical points and notice they are not isolated. What will 
happen to the population in the forest if it starts at some positive numbers. Hint: Think of 
the constant of motion. 


Exercise 5.3.6: The following system of differential equations models a pair of populations 
interacting. 


a) Does this system of differential equations better fit with a competing species model or 
a predator-prey model? If it is predator-prey, which species is the predator? 


b) Find and classify the critical point (if it exists) with both x > 0 and y > 0. 


c) Describe what is going to happen to the population of these species over time. It this 
depends on the initial condition, say so. 


Exercise 5.3.7: The following system of differential equations models a pair of populations 


interacting. 
dx dy 
— = 2(6 = 3y = 22 —=y(4-y-32 
U8 8y—22) = yl4—y — 82) 
a) Does this system of differential equations better fit with a competing species model or 
a predator-prey model? If it is predator-prey, which species is the predator? 


b) Find and classify the critical point (if it exists) with both x > 0 and y > 0. 


c) Describe what is going to happen to the population of these species over time. It this 
depends on the initial condition, say so. 


Exercise 5.3.8: The following system of differential equations models a pair of populations 
interacting. 


dx dy _ 


= 27(5 — x — 2y) i 


Oe a T= 8 
a y(7 — x — 3y) 


a) Does this system of differential equations better fit with a competing species model or 
a predator-prey model? If it is predator-prey, which species is the predator? 


b) Find and classify the critical point (if it exists) with both x > 0 and y > 0. 


c) Describe what is going to happen to the population of these species over time. It this 
depends on the initial condition, say so. 


Exercise 5.3.9: 
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a) Suppose x and y are positive variables. Show - attains a maximum at (1,1). 


b) Suppose a, b,c,d are positive constants, and also suppose x and y are positive variables. 
ad 


Show 4+, attains a maximum at (4c, 4/b). 


Exercise 5.3.10: Suppose that for the pendulum equation we take a trajectory giving the 


spinning-around motion, for example w = V 2 cos 6 + 29 +w. This is the trajectory where 


the lowest angular velocity is wi. Find an integral expression for how long it takes the 
pendulum to go all the way around. 


Exercise 5.3.11: Consider a predator-prey interaction where humans have gotten involved. 
The idea is that at least one of the species is valuable for food or another resource, and the 
two species still intact in their normal predator-prey manner. The first version of this will 
deal with “constant effort harvesting,” which means that humans will remove animals from 
the populations are a rate proportional to the population. This results in equations of the 


form 
dx dy 


He 7 tla — by — By) ae Ua t c& — Ee) 
where E, and E denote the amount of harvesting done. 
a) There is a single equilibrium solution with x > 0 and y > 0 in the case of no harvesting, 
that is, E, = Ey = 0. Find this equilibrium solution. 


b) Without doing any mathematical work, what do you think will happen to the equilibrium 
solution if just the prey is harvested? What if just the predator is harvested? What if 
both are harvested? 


c) Find the location of the equilibrium system in each of the three cases in the previous 
part. Do this in terms of the constants E, and Ey for all three cases. 


Exercise 5.3.12: The second version of this will deal with “constant yield harvesting,” 
which means that humans will remove animals from the populations at a fixed rate, no matter 
their population. This results in equations of the form 


dx dy _ 


em tle — by) — qe Ya + ct) — He 


where H, and H2 denote the amount of harvesting done. 


a) There is a single equilibrium solution with x > 0 and y > 0 in the case of no harvesting, 
that is, H, = Hj =0. Find this equilibrium solution. 


b) Without doing any mathematical work, what do you think will happen to the equilibrium 
solution if just the prey is harvested? What if just the predator is harvested? What if 
both are harvested? 


c) Find the location of the equilibrium system in each of the three cases in the previous 
part. Do this in terms of the constants H, and Hg for all three cases. 
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Exercise 5.3.13: The general competing species model has the form 


dx 


ee r(p1 — yy — Miz) = y(p2 — You — Moy) 


oy 
dt 
where p indicates the growth rate, M is related to the carrying capacity, and y is connected 
to the interaction term. Assume that this model is being used to represent species A and B 
of fish living in a pond at time t, which is initially stocked with both species of fish. We want 
to analyze the behavior of this equation under different sets of coefficients. 


a) If p2/y2 > pi/M, and p2/M2 > pi/y1, show that the only equilibrium populations in 
the pond are no fish, no fish of species A, or no fish of species B. What happens for 
large values of t? 


b) If p,/M, > p2/y2 and pi/y1 > p2/M2, show that the only equilibrium populations in 
the pond are no fish, no fish of species A, or no fish of species B. What happens for 
large values of t? 


c) Suppose that p2/72 > p1/M, and p,/71 > p2/M2. Show that there is a stable equilibrium 
where both species coexist. 


Exercise 5.3.14 (challenging): Take the pendulum, suppose the initial position is 0 = 0. 


a) Find the expression for w giving the trajectory with initial condition (0,wo9). Hint: 
Figure out what C' should be in terms of wo. 


b) Find the crucial angular velocity w,, such that for any higher initial angular velocity, 
the pendulum will keep going around its axis, and for any lower initial angular velocity, 
the pendulum will simply swing back and forth. Hint: When the pendulum doesn’t go 
over the top the expression for w will be undefined for some Os. 


c) What do you think happens if the initial condition is (0,w ), that is, the initial angle is 
0, and the initial angular velocity is exactly wy. 
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5.4 Limit cycles 


Attribution: [JL], §8.4. 


After this section, you will be able to: 


e Identify differential equations that have limit cycles from slope fields and 
e Find and classify limit cycles of systems of differential equations by converting 
the system to depend on radius. 


For nonlinear systems, trajectories do not simply need to approach or leave a single point. 
They may in fact approach a larger set, such as a circle or another closed curve. 


Example 5.4.1: The Van der Pol oscillator* is the following equation 
x” — p(1—2*)2'+2=0, 


where ps is some positive constant. The Van der Pol oscillator originated with electrical 
circuits, but finds applications in diverse fields such as biology, seismology, and other physical 
sciences. 

For simplicity, let us use 4 = 1. A phase diagram is given in the left-hand plot in 
Figure 5.16. Notice how the trajectories seem to very quickly settle on a closed curve. On 
the right-hand side is the plot of a single solution for t = 0 to t = 30 with initial conditions 
x(0) = 0.1 and z’(0) = 0.1. The solution quickly tends to a periodic solution. 
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Figure 5.16: The phase portrait (left) and a graph of a sample solution of the Van der Pol 
oscillator. 


The Van der Pol oscillator is an example of so-called relaxation oscillation. The word 
relaxation comes from the sudden jump (the very steep part of the solution). For larger yu 


*Named for the Dutch physicist Balthasar van der Pol (1889-1959). 
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the steep part becomes even more pronounced, for small jy the limit cycle looks more like a 
circle. In fact, setting w = 0, we get x” + x = 0, which is a linear system with a center and 
all trajectories become circles. 


What we see in this example is a curve to which many solution seem to head towards as t 
gets larger. This motivates the following definition. 


Definition 5.4.1 


1. A trajectory in the phase portrait that is a closed curve (a curve that is a loop) 
is called a closed trajectory. 


2. A limit cycle is a closed trajectory such that at least one other trajectory spirals 
into it. 


3. If all trajectories that start near the limit cycle spiral into it, the limit cycle is 
called asymptotically stable. 


For example, the closed curve in the phase portrait for the Van der Pol equation is a limit 
cycle, and the limit cycle in the Van der Pol oscillator is asymptotically stable. 

Given a closed trajectory on an autonomous system, any solution that starts on it is 
periodic. Such a curve is called a periodic orbit. More precisely, if (x(t), y(t)) is a solution 
such that for some to the point (x(to), y(to)) lies on a periodic orbit, then both x(t) and y(t) 
are periodic functions (with the same period). That is, there is some number P such that 
x(t) =a(t+ P) and y(t) = y(t + P). 

We would like to be able to identify when these sorts of periodic orbits can or can’t 
happen to understand more about these systems. Thankfully, we have a theorem that gives 
us some help here. 


Theorem 5.4.1 (Poincaré—Bendixson) 


Consider the system 
a=f(z,y), y =9(2,y), (5.9) 


where the functions f and g have continuous derivatives in some region R in the plane. 


Suppose F is a closed bounded region (a region in the plane that includes its boundary 
and does not have points arbitrarily far from the origin). Suppose (x(t), y(t)) is a 
solution of (5.9) in R that exists for all t > to. Then either the solution is a periodic 
function, or the solution tends towards a periodic solution in R. 


The main point of the theorem*is that if you find one solution that exists for all t large 
enough (that is, as t goes to infinity) and stays within a bounded region, then you have found 
either a periodic orbit, or a solution that spirals towards a limit cycle or tends to a critical 
point. That is, in the long term, the behavior is very close to a periodic function. Note that 
a constant solution at a critical point is periodic (with any period). The theorem is more a 
qualitative statement rather than something to help us in computations. In practice it is hard 


*Tvar Otto Bendixson (1861-1935) was a Swedish mathematician. 
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to find analytic solutions and so hard to show rigorously that they exist for all time. But if 
we think the solution exists we numerically solve for a large time to approximate the limit 
cycle. Another caveat is that the theorem only works in two dimensions. In three dimensions 
and higher, there is simply too much room. 

The theorem applies to all solutions in the Van der Pol oscillator. Solutions that start at 
any point except the origin (0,0) will tend to the periodic solution around the limit cycle, 
and if the initial condition of (0,0) will lead to the constant solution x = 0, y = 0. 


Example 5.4.2: Consider 
xe =y+(e2+y?—1)2, y= —2+(2? + y? —1)’y. 


A vector field along with solutions with initial conditions (1.02, 0), (0.9,0), and (0.1,0) are 
drawn in Figure 5.17. Analyze this system to determine what will happen to the solution for 
a variety of initial conditions. 
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Figure 5.17: Semistable limit cycle example. 


Solution: Notice that points on the unit circle (distance one from the origin) satisfy x? + 
y? —1=0. And z(t) = sin(t), y = cos(t) is a solution of the system. Therefore we have a 
closed trajectory. For points off the unit circle, the second term in x’ pushes the solution 
further away from the y-axis than the system x’ = y, y’! = —a, and y’ pushes the solution 
further away from the z-axis than the linear system 2’ = y, y' = —x. In other words for all 
other initial conditions the trajectory will spiral out. 

This means that for initial conditions inside the unit circle, the solution spirals out towards 
the periodic solution on the unit circle, and for initial conditions outside the unit circle the 
solutions spiral off towards infinity. Therefore the unit circle is a limit cycle, but not an 
asymptotically stable one. In relation to the terms used for autonomous equations in § 1.7, 
we could refer to this as a semistable limit cycle, since on one side (inside) the solutions 
spiral towards the periodic orbit, while on the other side (outside) the solutions move away. 
The Poincaré—Bendixson Theorem applies to the initial points inside the unit circle, as those 
solutions stay bounded, but not to those outside, as those solutions go off to infinity. | 
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A very similar analysis applies to the system 
w=yt(a ty —Va, y= et (a +y°—1)y. 


We still obtain a closed trajectory on the unit circle, and points outside the unit circle spiral 

out to infinity, but now points inside the unit circle spiral towards the critical point at the 

origin. So this system does not have a limit cycle, even though it has a closed trajectory. 
One way to see this more explicitly is by trying to write this all in terms of 


r=V/f/zr*+y". 
For simplicity here, we will determine everything in terms of 
sa=raa+ y” 


because as long as r > 0, r and s always have the same behavior (in terms of increasing and 
decreasing), and it is easier to compute with s. 
Using the first example 


ga=yt(ety - 1)*x, y =—e+ (+7 - 1)*y. 
we see that 
s' = Qara’ + 2yy’ 
= 2a(y + (2? + y? — 1)'n + 2y(—2 + (a? + y? - 1)°y) 


= Way + 22° (2? +7? — 1)? — Qry + 2y? (2? +y? — 1) 
s’ = 2s(s — 1)? 


Thus, we are left with the equation 


“ = 2s(s—1)° 

which is an autonomous first-order equation that we can analyze. We have two equilibrium 
solutions in terms of s at s = 0, which corresponds to the origin, and s = 1, which corresponds 
to the unit circle. We can then plug in values to see that for s = x, a > 0, so that the 
solutions will increase out to the unit circle. For s > 1, ds > 0 as well, so solutions move away 
from the circle outside it. This is the same as the result we obtained in the first example. 


For the second example, we end up with the autonomous equation 


which is negative for 0 < s < 1 and positive for 1 < s, giving that solutions that start inside 
the unit circle will converge to the origin, and solutions that start outside the circle will move 
away from it. 

Due to the Picard theorem (Theorem 4.1.1 on page 278) we find that no matter where we 
are in the plane we can always find a solution a little bit further in time, as long as f and g 
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have continuous derivatives. So if we find a closed trajectory in an autonomous system, then 
for every initial point inside the closed trajectory, the solution will exist for all time and it 
will stay bounded (it will stay inside the closed trajectory). Since the closed trajectory is a 
solution, we can not cross it (by Picard theorem), and so we have to stay trapped inside. So 
the moment we found the solution above going around the unit circle, we knew that for every 
initial point inside the circle, the solution exists for all time and the Poincaré—Bendixson 
theorem applies. 


Let us next look for conditions when limit cycles (or periodic orbits) do not exist. We 
assume the equation (5.9) is defined on a simply connected region, that is, a region with no 
holes we can go around. For example the entire plane is a simply connected region, and so is 
the inside of the unit disc. However, the entire plane minus a point is not a simply connected 
domain as it has a “hole” at the origin. 


Theorem 5.4.2 (Bendixson—Dulac) 


Suppose R is a simply connected region, and the expression® 


Ques 
Ox Oy 


is either always positive or always negative on R (except perhaps a small set such as 
on isolated points or curves) then the system (5.9) has no closed trajectory inside R. 


Usually the expression in the Bendixson—Dulac Theorem is Hef 4 29) for some continuously 
fa) Oy 


differentiable function y. For simplicity, let us just consider the case y = 1. 


The theorem* gives us a way of ruling out the existence of a closed trajectory, and hence a 
way of ruling out limit cycles. The exception about points or curves means that we can allow 
the expression to be zero at a few points, or perhaps on a curve, but not on any larger set. 


Example 5.4.3: Let us look at x’ = y+ ye”, y’ =z in the entire plane (see Example 5.1.4 
on page 377) and try to apply Theorem 5.4.2. 


Solution: The entire plane is simply connected and so we can apply the theorem. We 
compute ae + a = y’e* +0. The function y’e” is always positive except on the line y = 0. 
Therefore, via the theorem, the system has no closed trajectories. | 


In some books (or the internet) the theorem is not stated carefully and it concludes there 
are no periodic solutions. That is not quite right. The example above has two critical points 
and hence it has constant solutions, and constant functions are periodic. The conclusion 
of the theorem should be that there exist no trajectories that form closed curves. Another 
way to state the conclusion of the theorem would be to say that there exist no nonconstant 
periodic solutions that stay in R. 

Let us look at a somewhat more complicated example. 


Example 5.4.4: Take the system 2’ = —y—2?, y'! = —r+y? (see Example 5.1.3 on page 376) 
and look at how Theorem 5.4.2 works here. 


*Henri Dulac (1870-1955) was a French mathematician. 
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Solution: We compute oe + oa = —27+2y = 2(—xr+y). This expression takes on both signs, 


so if we are talking about the whole plane we cannot simply apply the theorem. However, we 
could apply it on the set where —2+ y > 0. Via the theorem, there is no closed trajectory in 
that set. Similarly, there is no closed trajectory in the set —x + y < 0. We cannot conclude 
(yet) that there is no closed trajectory in the entire plane. Perhaps half of it is in the set 
where —x + y > 0 and the other half is in the set where —x + y < 0. 

The key is to look at the line where —x + y = 0, or x = y. On this line 2! = —y — 2? = 
—¢—2* and y’ = —x+y? = —x +2’. In particular, when z = y then 2’ < y’. That means 
that the arrows, the vectors (2’, y’), always point into the set where —z + y > 0. There is no 
way we can start in the set where —x + y > 0 and go into the set where —z + y < 0. Once 
we are in the set where —27 +y > 0, we stay there. So no closed trajectory can have points in 
both sets. | 


Example 5.4.5: Consider 2’ = y + (x7 + y? — 1)a, y! = —x 4+ (x? + y? — 1)y, and consider 
the region R given by x? + y? > s. That is, R is the region outside a circle of radius B 
centered at the origin. Then there is a closed trajectory in R, namely x = cos(t), y = sin(t). 
Furthermore, 


<4 —* = dg? + Ay? — 2, 
xe 


which is always positive on R. So what is going on? The Bendixson—Dulac theorem does not 
apply since the region R is not simply connected—it has a hole, the circle we cut out! 


5.4.1 Exercises 


Exercise 5.4.1: Consider the two-dimensional system of differential equation written in 


polar coordinates as 
dr dé 
—S7relreare =] 
qo = Yee) Ti 

Determine all limit cycles, periodic solutions, and classify the stability of each of these 

solutions. 


Exercise 5.4.2: Consider the two-dimensional system of differential equation written in 
polar coordinates as 

db 
ao 
Determine all limit cycles, periodic solutions, and classify the stability of each of these 
solutions. 


=P (r=—1)7(r —3) 


Exercise 5.4.3:* Consider the system of differential equation given by 


dx 
ee — Dy? 
ht r(3 — 2y 


dy 

2 2 
—2£ —=y(3-y"). 

ht eS) 

Find and classify all limit cycles by converting to an autonomous equation in r = ,/x? + y? 


ors=27+y". 
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Exercise 5.4.4:* Consider the system of differential equation given by 


d d 
+ = —2(x° + y’)? + 62(a* + y”) — 8x + by a = —y(2? + y")? + 6y(2" + y*) — 8y — Ge. 


Find and classify all limit cycles by converting to an autonomous equation in r = ,/x? + y? 
er 
ors=ax4*+y". 


Exercise 5.4.5: Consider the system 


d 
att tale? + — 2 ry) 


d 
; = Ie + y + y(a? + y? — 2/2? +?) 


(5.10) 


’ oy, ' 
a) Use polar coordinates to write au? function of r. 


ds 


b) Draw the phase line of the DE i 


f(r), where f(r) is the function from part a. 


c) Does the system (5.10) have a limit cycle? If so, find it. If not, explain why not. For 
each positive root of f(r), decide whether the corresponding trajectory one is stable, 
unstable, or semistable. 


Exercise 5.4.6: Show that the following systems have no closed trajectories. 
ajvi=a+y, y=yr+2’, be Se, -g Ser, 
c) aw =24+3y-y, yl =yi4+27. 

Exercise 5.4.7:* Show that the following systems have no closed trajectories. 


ajt=aty’, y=yte2’, b) «' =—asin?(y), y' =e’, 
j@=2y, yf S24 a. 


Exercise 5.4.8:* Suppose an autonomous system in the plane has a solution x = cos(t)+e‘, 


y =sin(t) +e. What can you say about the system (in particular about limit cycles and 
periodic solutions)? 


Exercise 5.4.9: Formulate a condition for a 2-by-2 linear system x’ = AZ to not be a center 
using the Bendixson—Dulac theorem. That is, the theorem says something about certain 
elements of A. 


Exercise 5.4.10: Explain why the Bendixson—Dulac Theorem does not apply for any 
conservative system x” + h(x) = 0. 


Exercise 5.4.11: A system such as x’ = x,y' = y has solutions that exist for all time t, yet 
there are no closed trajectories. Explain why the Poincaré—Bendixson Theorem does not 


apply. 
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Exercise 5.4.12:* Show that the limit cycle of the Van der Pol oscillator (for 1. > 0) must 
not lie completely in the set where —1 < x < 1. Compare with Figure 5.16 on page 413. 


Exercise 5.4.13: Differential equations can also be given in different coordinate systems. 
Suppose we have the system r’ = 1 —r?, 6’ = 1 given in polar coordinates. Find all the closed 
trajectories and check if they are limit cycles and if so, if they are asymptotically stable or 
not. 


Exercise 5.4.14:* Suppose we have the system r’ = sin(r), 6’ = 1 given in polar coordinates. 
Find all the closed trajectories. 
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5.5 Chaos 


Attribution: [JL], §8.6. 


After this section, you will be able to: 


e Identify chaotic behavior and how it is distinct from other types of equations. 


You have surely heard the idea of the “butterfly effect,” that the flap of a butterfly wing 
in the Amazon can cause hurricanes in the North Atlantic. In a prior section, we mentioned 
that a small change in initial conditions of the planets can lead to very different configuration 
of the planets in the long term. These are examples of chaotic systems. Mathematical chaos 
is not really chaos, there is precise order behind the scenes. Everything is still deterministic. 
However a chaotic system is extremely sensitive to initial conditions. This also means even 
small errors induced via numerical approximation create large errors very quickly, so it is 
almost impossible to numerically approximate for long times. This is a large part of the 
trouble, as chaotic systems cannot be in general solved analytically. 

Take the weather, the most well-known chaotic system. A small change in the initial 
conditions (the temperature at every point of the atmosphere for example) produces drastically 
different predictions in relatively short time, and so we cannot accurately predict weather. 
And we do not actually know the exact initial conditions. We measure temperatures at a few 
points with some error, and then we somehow estimate what is in between. There is no way 
we can accurately measure the effects of every butterfly wing. Then we solve the equations 
numerically introducing new errors. You should not trust weather prediction more than a 
few days out. 

Chaotic behavior was first noticed by Edward Lorenz* in the 1960s when trying to model 
thermally induced air convection (movement). Lorentz was looking at the relatively simple 
system: 

x’ = —10x + 10y, y = 28x — y — 22, z= ~ Fa + ry. 
A small change in the initial conditions yields a very different solution after a reasonably 
short time. 

A simple example the reader can experiment with, and which displays 

chaotic behavior, is a double pendulum. The equations for this setup are 
somewhat complicated, and their derivation is quite tedious, so we will not 
bother to write them down. The idea is to put a pendulum on the end of 
another pendulum. The movement of the bottom mass will appear chaotic. 
This type of chaotic system is a basis for a whole number of office novelty 
desk toys. It is simple to build a version. Take a piece of a string. Tie two 
heavy nuts at different points of the string; one at the end, and one a bit 
above. Now give the bottom nut a little push. As long as the swings are not 
too big and the string stays tight, you have a double pendulum system. 


*Edward Norton Lorenz (1917-2008) was an American mathematician and meteorologist. 
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5.5.1 Duffing equation and strange attractors 


Let us study the so-called Duffing equation: 
x” + aa’ + bx + cx? = C cos(wt). 


Here a, b, c, C, and w are constants. Except for the cv? term, this equation looks like a 
forced mass-spring system. The cx? means the spring does not exactly obey Hooke’s law 
(which no real-world spring actually does obey exactly). When c is not zero, the equation 
does not have a closed form solution, so we must resort to numerical solutions, as is usual for 
nonlinear systems. Not all choices of constants and initial conditions exhibit chaotic behavior. 
Let us study 

x” + 0.052’ + 2° = 8cos(t). 


The equation is not autonomous, so we cannot draw the vector field in the phase plane. 
We can still draw the trajectories. In Figure 5.18 we plot trajectories for t going from 0 to 
15, for two very close initial conditions (2,3) and (2, 2.9), and also the solutions in the (z,t) 
space. The two trajectories are close at first, but after a while diverge significantly. This 
sensitivity to initial conditions is precisely what we mean by the system behaving chaotically. 


Figure 5.18: On left, two trajectories in phase space for0 <t< 15, for the Duffing equation one 
with initial conditions (2,3) and the other with (2,2.9). On right the two solutions in (x, t)-space. 


Let us see the long term behavior. In Figure 5.19 on the next page, we plot the behavior 
of the system for initial conditions (2,3) for a longer period of time. It is hard to see any 
particular pattern in the shape of the solution except that it seems to oscillate, but each 
oscillation appears quite unique. The oscillation is expected due to the forcing term. We 
mention that to produce the picture accurately, a ridiculously large number of steps* had 
to be used in the numerical algorithm, as even small errors quickly propagate in a chaotic 
system. 


*In fact for reference, 30,000 steps were used with the Runge-Kutta algorithm, see exercises in § 1.6. 
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0 20 40 60 80 100 


Figure 5.19: The solution to the given Duffing equation for t from 0 to 100. 


It is very difficult to analyze chaotic systems, or to find the order behind the madness, 
but let us try to do something that we did for the standard mass-spring system. One way we 
analyzed the system is that we figured out what was the long term behavior (not dependent 
on initial conditions). From the figure above, it is clear that we will not get a nice exact 
description of the long term behavior for this chaotic system, but perhaps we can find some 
order to what happens on each “oscillation” and what do these oscillations have in common. 

The concept we explore is that of a Poincaré section*. Instead of looking at t in a certain 
interval, we look at where the system is at a certain sequence of points in time. Imagine 
flashing a strobe at a fixed frequency and drawing the points where the solution is during 
the flashes. The right strobing frequency depends on the system in question. The correct 
frequency for the forced Duffing equation (and other similar systems) is the frequency of the 
forcing term. For the Duffing equation above, find a solution (x(t), y(t)), and look at the 
points 


(x(0),y(0)), (w(2r),y(27)),  (w(4r),y(47)),  (w(6r), y(67)), 


As we are really not interested in the transient part of the solution, that is, the part of the 
solution that depends on the initial condition, we skip some number of steps in the beginning. 
For example, we might skip the first 100 such steps and start plotting points at t = 100(2z), 
that is 


(x(2007), y(2007)),  (a(2027), y(2027)), — (a(2047r), y(2047r)), 


The plot of these points is the Poincaré section. After plotting enough points, a curious 
pattern emerges in Figure 5.20 on the following page (the left-hand picture), a so-called 
strange attractor. 

Given a sequence of points, an attractor is a set towards which the points in the sequence 
eventually get closer and closer to, that is, they are attracted. The Poincaré section is not 
really the attractor itself, but as the points are very close to it, we see its shape. The strange 


*Named for the French polymath Jules Henri Poincaré (1854-1912). 
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Figure 5.20: Strange attractor. The left plot is with no phase shift, the right plot has phase shift 


m/4, 


attractor is a very complicated set. It has fractal structure, that is, if you zoom in as far as 
you want, you keep seeing the same complicated structure. 

The initial condition makes no difference. If we start with a different initial condition, the 
points eventually gravitate towards the attractor, and so as long as we throw away the first 
few points, we get the same picture. Similarly small errors in the numerical approximations 
do not matter here. 

An amazing thing is that a chaotic system such as the Duffing equation is not random at 
all. There is a very complicated order to it, and the strange attractor says something about 
this order. We cannot quite say what state the system will be in eventually, but given the 
fixed strobing frequency we narrow it down to the points on the attractor. 

If we use a phase shift, for example 7/4, and look at the times 


Ha, Qe+7/4, 4n4+7/4, 60+ 7/4, 


we obtain a slightly different attractor. The picture is the right-hand side of Figure 5.20. It 
is as if we had rotated, moved, and slightly distorted the original. For each phase shift you 
can find the set of points towards which the system periodically keeps coming back to. 

Study the pictures and notice especially the scales—where are these attractors located in 
the phase plane. Notice the regions where the strange attractor lives and compare it to the 
plot of the trajectories in Figure 5.18 on page 422. 

Let us compare this section to the discussion in § 2.6 about forced oscillations. Take the 
equation 


Ff 
x" + 2pa! + wer = — cos(wt). 
m 


This is like the Duffing equation, but with no x? term. The steady periodic solution is of the 
form 


x = Ccos(wt +7). 
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Strobing using the frequency w, we obtain a single point in the phase space. The attractor 
in this setting is a single point—an expected result as the system is not chaotic. It was the 
opposite of chaotic: Any difference induced by the initial conditions dies away very quickly, 
and we settle into always the same steady periodic motion. 


5.5.2 The Lorenz system 


In two dimensions to find chaotic behavior, we must study forced, or non-autonomous, systems 
such as the Duffing equation. The Poincaré-Bendixson Theorem says that a solution to an 
autonomous two-dimensional system that exists for all time in the future and does not go 
towards infinity is periodic or tends towards a periodic solution. Hardly the chaotic behavior 
we are looking for. 

In three dimensions even autonomous systems can be chaotic. Let us very briefly return 
to the Lorenz system 


/ 


8 
x’ = —10x + 10y, y = 28 —y-— «22, z= 732 + By. 


The Lorenz system is an autonomous system in three dimensions exhibiting chaotic behavior. 
See the Figure 5.21 for a sample trajectory, which is now a curve in three-dimensional space. 


Figure 5.21: A trajectory in the Lorenz system. 


The solutions tend to an attractor in space, the so-called Lorenz attractor. In this case no 
strobing is necessary. Again we cannot quite see the attractor itself, but if we try to follow a 
solution for long enough, as in the figure, we get a pretty good picture of what the attractor 
looks like. The Lorenz attractor is also a strange attractor and has a complicated fractal 
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structure. And, just as for the Duffing equation, what we want to draw is not the whole 
trajectory, but start drawing the trajectory after a while, once it is close to the attractor. 

The path of the trajectory is not simply a repeating figure-eight. The trajectory spins 
some seemingly random number of times on the left, then spins a number of times on the 
right, and so on. As this system arose in weather prediction, one can perhaps imagine a few 
days of warm weather and then a few days of cold weather, where it is not easy to predict 
when the weather will change, just as it is not really easy to predict far in advance when 
the solution will jump onto the other side. See Figure 5.22 for a plot of the x component 
of the solution drawn above. A negative x corresponds to the left “loop” and a positive x 
corresponds to the right “loop”. 

Most of the mathematics we studied in this book is quite classical and well understood. 
On the other hand, chaos, including the Lorenz system, continues to be the subject of current 
research. Furthermore, chaos has found applications not just in the sciences, but also in art. 


0.0 25 5.0 75 10.0 125 15.0 


Figure 5.22: Graph of the x(t) component of the solution. 


5.5.3. Exercises 


Exercise 5.5.1 (*): Find critical points of the Lorenz system and the associated lineariza- 
tions. 


Exercise 5.5.2: For the non-chaotic equation x" + 2p2' + wer = fn cos(wt), suppose we 
strobe with frequency w as we mentioned above. Use the known steady periodic solution to 
find precisely the point which is the attractor for the Poincaré section. 


Exercise 5.5.3 (project): Construct the double pendulum described in the text with a 
string and two nuts (or heavy beads). Play around with the position of the middle nut, and 
perhaps use different weight nuts. Describe what you find. 
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Exercise 5.5.4 (project): A simple fractal attractor can be drawn via the following chaos 
game. Draw the three vertices of a triangle and label them, say p,, pz and p3. Draw some 
random point p (it does not have to be one of the three points above). Roll a die to pick 
of the pi, p2, or p3 randomly (for example 1 and 4 mean p,, 2 and 5 mean po, and 3 and 6 
mean p3). Suppose we picked po, then let Pnew be the point exactly halfway between p and 
p2. Draw this point and let p now refer to this new point Pnew. Rinse, repeat. Try to be 
precise and draw as many iterations as possible. Your points will be attracted to the so-called 
Sierpinski triangle. A computer was used to run the game for 10,000 iterations to obtain the 
picture in Figure 5.23. 
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Figure 5.23: 10,000 iterations of the chaos game producing the Sierpinski triangle. 


Exercise 5.5.5 (computer project): Use a computer software (such as Matlab, Octave, or 
perhaps even a spreadsheet), plot the solution of the given forced Duffing equation with 
Euler’s method. Plotting the solution for t from 0 to 100 with several different (small) step 
sizes. Discuss. 
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Appendix A 
Introduction to MATLAB 


This document is meant to provide a review of some of the main skills and techniques in 
MATLAB that are necessary to complete the various MATLAB assignments throughout the 
course. In addition, these skills will be useful when attempting to use MATLAB, both for 
illustrating problems in differential equations and for solving other types of problems that 
can be analyzed using this software. 


A.1l The MATLAB Interface 


There are many components to the MATLAB interface, and the way that the window is 
organized can be fully customized. There are four main components of this interface. 


1. Current Folder window. This shows the current folder in which MATLAB is running. 
This determines what files that MATLAB currently has access to and what functions 
and methods can be called. 


2. Editor window. This is the main code-editing window, where script files can be written, 
edited, saved, and run. 


3. Command window. This is where individual lines of code can be entered to see how 
they work. 


4. Workspace window. This shows a list of all variables that currently exist, as well as 
their values or sizes. 


All four of these components are very useful in organizing thoughts and programming 
practices while using MATLAB. Both the Default layout and Two-Column layout (as of 
MATLAB R2019b) contain all four of these windows in different locations. Either of these 
will work for programming in MATLAB, as well as any modifications of them. The current 
format can be saved using Layout - Save Layout if needed. 


A.1.1 File Structure 


The main type of file used in MATLAB is the Script file. These are saved as ‘*.m’ files 
and can represent both stand-alone executable files and functions that can be called from 
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Command Window 


Select ileto view details 


Figure A.1: The default layout provided by MATLAB. 


other scripts. For running simple, one-line expressions or debugging code, the Command 
Window and the command line prompt can be useful. However, for anything more involved 
and complicated than that, the script editor should be used instead. 

In writing a script file or using the Command window, the Current Folder window shows 
all of the files in the current directory. These are all of the files that MATLAB has access 
to while running a MATLAB file that it saved in that folder. This means that if a script 
wants to call a method, it either needs to be a built-in method or a function file that is 
contained within the same script file or the Current Folder. For more information about 
writing functions, see Section A.4. 

To use script files, multiple lines of code can be entered in a row, and MATLAB will 
execute them in sequence when the “Run” button is clicked. This button is in the “Editor’ 
tab at the top of the screen. 
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Figure A.2: Location of the Run buttons on the MATLAB interface. 


MATLAB Live Scripts can also be used to do very similar things, with some additional 
benefits. These allow the MATLAB code to be viewed side-by-side with the output, as 
well as an easy export to PDF functionality. These are saved as ‘*.mlx’ files. These 
work the same way as scripts in terms of how code is written, and allow the user to 
mix between text (which can be resized and formatted) and code. For more information 
on Live Scripts, see the website https: //www.mathworks.com/help/matlab/matlab_prog/ 
what-is-a-live-script-or-function.html. 

Live Scripts also have the ability to put section breaks between different pieces of code 
and then run individual sections using the “Run Section” button at the top of the editor. 


A.2. COMPUTATION IN MATLAB 43 


e 


With Live Scripts, it is necessary to run the entire code (by clicking the run button) before 
exporting as a PDF in order to get the correct images and outputs in the final PDF. To 
export, go to Save at the top of the screen, click the down arrow under it, and select “Export 
to PDF” after running the code to regenerate all of the images. 
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Figure A.8: Header Bar for the MATLAB Live Script Interface. 


A.2 Computation in MATLAB 


MATLAB can do many of the simple computational operations that would be expected from 
a calculator. It is easiest to see these operations by using the Command Window, but they 
can also be implemented in scripts if desired. Addition and subtraction work in standard 
ways. In the command line, typing 


PoP) 


and pressing ENTER will give an output of 


ans = 
5 


showing the answer of this computation. For any computation or line of code, putting a 
semi-colon (;) at the end will suppress the output, in that typing 


Ban oie 


will not show any output. However, MATLAB did do the computation, which can be 
shown by storing this output in a variable and doing something with it later. 

Multiplication and division, and by extension powers, can work differently in MATLAB. 
As MATLAB is built around using matrices for calculations and is optimized for this approach, 
the program interprets all multiplication, division, and exponentiation in terms of matrices 
as a default. Both components of the multiplication are simple scalars (numbers), then this 
is fine. The ‘*’ symbol works for multiplication in this context: 


>> 4*6 
ans = 
24 


as well as using ‘/’ for division and ‘’ for exponentiation. Issues may arise when the code 
wants to compute products or powers of multiple values at the same time. Many MATLAB 


432 APPENDIX A. INTRODUCTION TO MATLAB 


built-in functions will automatically combine multiple of the same type of calculation into 
a ‘vectorized’ calculation, where if the code wanted to compute the sum of two numbers a 
bunch of times, it would put all of these numbers into arrays and then add the two vectors 
together. This completes the task of adding all of the different pairs of numbers together, but 
saves time by not doing them all individually. This works great for addition and subtraction, 
because addition and subtraction of arrays or matrices is done element-wise, which is the 
exact operation we wanted to compute in the first place. 

However, mutliplication is different. Matrix multiplication is a different operation that, 
in particular, is not element-wise multiplication. Beyond that, even if two matrices are the 
same size, it is possible that their product, in the normal matrix sense, is not defined. In 
MATLAB, the product 


tal Sa] ee) (4 Sh Dlg 


will return an error because the matrices are not the correct size. From a human point 
of view, the output desired from this code was likely [4 6 6], the product of each term 
individually. To obtain this in MATLAB, we need the elementwise operations ‘.*’, ‘./’ and ‘.~’ 
for multipication, division, and exponentiation, respectively. Thus, the following computations 
can be made in MATLAB 


SS fl 2 8] .e [As 2) 
ans = 

[4 6 6] 
Se [Fl -4 el). 2 


ans = 

[1 16 36] 
SS [by 42] .f LO 2 6) 
ans = 

[0.5 2 0.3333] 


There are many built-in functions in MATLAB that can help with computation and 
algebra. 


e sqrt (x) will compute the square root of a number 2. 

e exp(x) will compute e” for e the base of the natural logarithm, and x any number. 
Note that MATLAB does not know the definition of e built-in, so it will either need to 
be defined (using exp(1)) or just use exp() whenever it is needed. 


e abs(x) computes the absolute value of a number 2. 


e log(x) computes the natural logarithm of a number x. The functions log2 and log10 
compute the log base 2 and log base 10 respectively. 


e Trigonometric functions can also be computed with sin(x), cos(x), and tan(x). 
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A.3 Variables and Arrays 


As with other programming languages, MATLAB utilizes variables to store information and 
use it later. The name of variables in MATLAB must start with a letter, but the rest of the 
name can consist of letters, digits, or underscores. Variables should be named suggestively 
corresponding to what this information is or the way it will be used. Variables do not need to 
be created in advance, they are created when something is stored in the variable by putting 
the name on the left side of an equals sign, with the computation that gives rise to that 
variable on the right. Even though the output is suppressed, the line 


em 


will store the value 5 in the variable val, where it can be used later. For example, 


>> val *« 4 
ans = 
20 


Ss Wel) ap 2 
ans = 
27 


However, trying to use a variable name without defining it first will cause MATLAB to 
give an error: 


22 0 
Undefined function or variable 'r'. 


As variables do not need to be created or instantiated before they are used, any variable 
can store any type of information. Two of the most common ones are numbers (double 
precision) or strings. 


numVar = sqrt(15); 


strVar ~*Hello World!''; 


Strings can be stored using either single or double quotes. Strings also have a lot of useful 
operations that can be used to make some MATLAB programs run more simply, but they 
are beyond the scope of this introduction. For information about what can be done with 
strings, see the MATLAB documentation https: //www.mathworks.com/help/matlab/ref / 
string.html. 

Another common variable data type that MATLAB is very comfortable with is arrays. 
As described previously, MATLAB defaults to matrices when considering multiplication and 
exponentiation operations. Arrays can be created using square brackets, with either spaces 
or commas between the entries. 
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These create horizontal arrays. Vertical arrays can also be created using semi-colons 
between each entry, and these can be combined with horizontal arrays to create a matrix, or 
rectangular array of values. 


= [5;7;81; 
[25335657 ls 


In these examples, A and B will be row arrays (or row vectors) with 3 elements, C 
will be a column vector with 3 elements, and M will be a matrix with two rows and three 
columns. For most situations that don’t involve matrices, row and column vectors will work 
equivalently, so either one can be used. Once matrices are involved, it matters which one is 
chosen, because MATLAB will multiply matrices and vectors in the same way that would be 
carried out mathematically, which means the dimensions need to match. 

To access elements of a matrix, parentheses are used. Unlike other programming languages, 
MATLAB starts indexing elements at 1, not zero. That is, with the above variables C(2) = 
7, since 7 is the second element of the array C’. In terms of accessing elements of matrices, 
the first index is the row and the second is the column. 


SS tes f.9,8°5,6,7¢ 
>> M(1,1) 
ans 


>> M(1,3) 
ans = 
3 


>> M(2,1) 
ans = 
5 


The matrix (and vectors) do have limits on how big they are, and attempting to access 
an element outside of that range will cause MATLAB to give an error. 


>> M(3,1) 


Index in position 1 exceeds array bounds (must not exceed 2). 


Among many other possible variables, another type that can be stored is a handle to a 
function. How to use functions will be described in Section A.4. The fact that all of these 
different data types can be stored in variables, with no real indication as to which type a 
given variable is, means it is critical to name variables carefully with what they correspond 
to. 
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A.4 Functions and Anonymous Functions 


A key component to programming in MATLAB is the idea of functions. These are program- 
ming objects that will accept a number of inputs (called arguments) and perform a given set 
of operations on those arguments, returning some set of ouputs back to the main program. 
These are mainly used to group code together that has a given purpose and can be called to 
carry out that purpose on a variety of outputs. An example of a built-in function like this is 
sum(V). This function takes in a linear array and will return the number that is the sum of 
all of the elements in the array (if the array is multi-dimensional, it will only sum along one 
dimension). This is a piece of code that could be written fairly easily; it would just involve 
taking the array, looping through it and adding up the value at each index. However, putting 
it into a function allows it to be called more simply in one line, allowing the main script to 
focus on the task at hand. 


There are two main ways that functions can be written in MATLAB. Functions can either 
be written at the bottom of the MATLAB script where they will be used or they can be 
written in their own separate script file. If written in a separate file, there can only be one 
function in each file, and the name of the file (once saved) must match the name given to the 
function. To write a function, the reserved word ‘function’ is used: 


function [a,b] = testFunction(x, y, z) 
4 Code here 


end 


Note: If this is done in a script by itself, the function line must be the first line of the 
code. There can be no code or comments above this line. 


In this case, the function takes in three inputs and returns two outputs. When writing 
the code inside the function, the three inputs will be called x, y, and z, and in order to tell 
the program what to send back to wherever this function was called, those outputs should be 
stored in variables a and b. For example, a function that takes in three numbers and returns 
their sum in the first output and the product in the second would look like 


function [a,b] = testFunction(x, y, z) 
a = xt+yt+z; 


b = x*y*zZ; 


end 


and that would work just fine. However, if any other MATLAB methods were going to 
use this function, there is a chance they would try to pass in array inputs. If so, then there 
would be an error in computing b, because those products would not be defined. The easiest 
way to fix this would be to use element-wise products, giving a function that looks like 
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function [a,b] = testFunction(x, y, z) 


X.*Y.*Z; 


These functions can be as complicated as necessary, including graphs, loops, calls to 
other functions, and many different components. However, if the function needed is a simple 
mathematical function, then this can be written in an shorter way with anonymous functions. 
For example, if the function f(x,y) = x? + 4xry + y? needed to be coded, it could be written 


as 
i S Gey) HO sr Aoeee tay cP Wy. Be 


and this will now make f a handle to the function that does exactly what is desired. If a 
later line of code is 


SS 2 (@, il) 
ans = 
13 


the function value will be computed at the desired point. Notice the use of element-wise 
operations again in this function definition to ensure that it will also work on array inputs. 
This works for these simple kinds of functions, and can be easier than adding an entire new 
function to the script file. 

Overall, the following two function definitions are almost equivalent. 


fShort = Q(x,y) x.°2 + y.°2; 


function z = fLong(x,y) 


B= 6. 2B a Wo 28 
end 


The only difference arises when trying to use these functions in built-in or written methods 
that require a handle to a function. The ‘@’ symbol at the beginning of the anonymous 
function indicates that the thing being defined (fShort) is a handle to a function that takes 
two inputs and computes an output from it. On the other hand, the definition of fLong is 
a function that does this, and is not a handle to that function. To fix this, an ‘Q@’ symbol 
needs to be put in-front of fLong before using it in one of these methods. As an example 
ode45 is a method that numerically computes the solution to a differential equation, and it 
requires a function handle in the first argument. So, the code 


ode45(fShort, [0, 3], 1) 


runs fine. However, 
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ode45(fLong, [0, 3], 1) 


throws an error about there being not enough inputs for fLong. This is because whenever 
MATLAB sees fLong, it is expecting to see two inputs next to it. This is not the case for 
fShort because of the way it was defined. To remedy this, the code needs to be written 


ode45(@fLong, [0, 3], 1) 


and then it will execute the same as the first line. 
With any of these functions, it is possible to restrict variables and get new functions. This 
can be fairly easily done with the same setup as for anonymous functions. The line of code 


fNew = @(y) fShort(1,y) 


will create a new handle for a function of one variable that is fShort when the x value is 
fixed to be 1. The exact same code will work for fLong as you are giving it two inputs. 


A.5 Loops and Branching Statements 


The code written in a MATLAB script will always proceed in order from one line to the next 
unless there is some alteration to the flow using loops or branching (if) statements. 


A.5.1 For Loops 


For loops are a form of iterative programming, where MATLAB will run the same bit of code 
multiple times with an iterative parameter that can change certain things about the code. If 
there is an element of the program that needs to carry out a process several times in a row, 
particularly using the previous step to compute the one after it, a for loop might be the best 
structure to use. A sample for loop has the following form: 


12One Goeiaecie = i213 10) 
4 CODE HERE 


end 


In this line, counter is the variable that is getting incremented over the list. The rest of 
that line says that counter starts at 1, increments by 1 each loop, and stops after 10. A line 
of the form counter = 2:5:34 will start at 2, increment by 5 each loop, and stop once the 
counter gets above 34, so after the iteration when counter = 32. 

In order to loop through an array of values, it is useful to figure out the size of the array 
and use that to determine how many times the loop should be run. This sort of programming 
will allow your code to work for a variety of different inputs, no matter the size. This can be 
done with code like this. 


438 APPENDIX A. INTRODUCTION TO MATLAB 


v= [p23 4,513 4 Mees Weel We Votre Lise Oj Walwes 
for counter = 1:1:length(v) 


v (counter) ~2 


To find how many elements are in an array, the length function will work for a linear 
array. If the array is more complicated, the size function can be used. This will give a list 
of values saying how large the array is in each dimension. 

MATLAB also has while loops, which allow a loop to run up until a condition becomes 
false. This is better than for loops in specific situations, but either one can be used. For the 
code developed here, for loops will be just as easy to write as while loops. 


A.5.2 If Statements 


If statements, or conditional statements, allow certain parts of code to be executed only if a 
certain condition is met. For instance, something like 


if counter < 5 
4, CODE HERE 


end 


will only execute if the counter is less than 5, and 


if mod(counter,2) == 
4 CODE HERE 


end 


will only run if counter is even, that is, if the remainder when dividing counter by 2 is zero. 
Notice that == is used for comparison here to check if two things are equal, while = is used 
for variable assignment. The condition part of an if statement can be anything that gives 
back a true or false result. For math operations, these can be any inequalities (<, <, >, >) 
or == for testing inequality. The operator ~ is used for “not”, in that a ~= b will be true 
if a is not equal to b, and false if they are the same. Outside of numbers, there are other 
MATLAB methods that will give true or false answers. These can be things like comparing 
strings, but this is beyond the code developed here. 


A.6 Plotting in MATLAB 


Graphing in MATLAB always involves plotting a set of points, but these can be fairly easily 
generated from functions as well. For example 
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eins S [fl 258),4,)5l 2 
ie = CG) s¢,°2 <2 Ds 
yPts = 285258, 5l]] 8 


figure(1); 

plot(xPts, yPts); 
figure(2); 

plot(xPts, fx(xPts)); 


4 @ Figure 2 > Eh + 
File Edit View Insert Tools Desktop Window Help ~| File Edit View Insert Tools Desktop Window Help 


rt 
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Figure A.4: Output from MATLAB plotting two graphs. 


will generate two figures, referred to by the lines figure(1) and figure(2), and allow the 
two graphs to be simultaneously drawn without overlapping each other. Any time MATLAB 
draws a plot (with the plot command) it will overwrite any plot that is already on the target 
figure. In order to put multiple plots on the same figure, the hold on; and hold off; 
commands can be used. 


xPts = linspace(1,5,100); 
i> Cl © 2 > ae ee 

(ee 2 OG) %.°2 = Slee + 1s 
figure(1); 


hold on; 
plot(xPts, fx(xPts)); 
plot(xPts, gx(xPts)); 
hold off; 


The linspace generates a list of 100 equally spaced values between 1 and 5 for plotting 
purposes. It gives an easy way to generate a lot of input values for plotting a smooth-looking 
graph. It also emphasizes the need to use the element-wise operations in these functions to 
make sure they all compute correctly. 

There are many additional options that can be passed to the plot method in order to 
change the color, shape, and size of the plot. For these options, refer to the MATLAB 
documentation on the plot function at https: //www.mathworks.com/help/matlab/ref/ 
plot.html. 
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Figure A.5: Output from MATLAB plotting two functions on the same axes. 


A.7 Supplemental Code Files 


There are eleven supplemental code files provided. In order to use these files in a script or a 
Live Script, they must be placed in the same folder as the script file, so that the Current 
Folder window contains both the file being executed and all of these function files. Another 
option would be to store all of these function files in a single folder, navigating to that folder 
in the MATLAB Current Folder window, right-clicking on the folder, and selecting “Add 
to Path.” The first of these is more recommended, but the second can also work if there 
is a common repository to store all of the users custom MATLAB functions. The function 
headers are given below along with a brief description of their use. 


function quiver244(f, t_min, t_max, y_min, y_max, col) 
hk quiver244.m 
4 Author: Matt Charnley 


This function draws a quiver plot for the ODE dy/dt = f(t,y) for 
t_min <= t <= t_maxz and y_min <= y <= y_max. The function f should be 

f, passed in as an anonymous function, of two variables or as a function 
handle 


The function draws this quiver plot in color col and saves it on the 
current figure, and generates a normalized version 
(all vectors are the same length) as the next figure, 
so that it can be accessed outside of this function. 
4, For this second figure, the magnitude of the arrows does not mean 


Z anything, but tt is easter to see the direction of them. 
so that it can be accessed outside of this function. It will start with 
hold on; and end with hold off;, so the figure needs to be cleared in the 


main file if needed. 
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The main point of this function is to simplify the process of drawing quiver plots. The code 
here takes care of the difficulties that arise from the built-in quiver function in MATLAB 
and allows the user to input the right-hand side of a first order ODE and generate quiver 
plots. It will draw a quiver plot in the first figure, and a normalized quiver plot (all vectors 
the same length) in the second figure. It can sometimes be easier to see the general trajectory 
of solutions from the normalized figure, so both graphs are provided. All of the plotting 
commands use the hold commands so that they will not overwrite anything on the desired 
figures. This allows the overlaying of multiple plots, but means that the code calling this 
method must clear the figure if it needs to be cleared. 


This code can be used as 
fi 10( bay be expon)s 


quiver244(f, 0, 5, -6, 6, 'b'); 


quiver244(@f2, 0, 5, -6, 


function z = f2(t,y) 
Z=t - exp(y); 
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Figure A.6: Sample output from the quiver244 function. 


In each case, the ‘b’ indicates that the quiver plot will be drawn in blue, and the 1 before 
that indicates that the two plots will be drawn on figures 1 and 2. 
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function samplePlots244(f, t_min, t_max, y_min, y_max, t_0, yar col) 

4 This function takes the ODE dy/dt = f(t,y) and plots sample solutions 

fh with initial value (t_0, y_0). It uses ode45 to sketch out the solutions. 
4 t_O must be between t_min and t_maz. It also truncates the function f so 

4 that functions will not go off to infinity, causing this to work properly 
fh on vector inputs for initial conditions in y. The input y_O can be a 


4 vector 

fh of wnitial values, and this function will plot a curve 

h for each of those values. If using a vector of initial 

4 conditions, the function must be written with vector element-wise 
A operations. 


This function follows the same setup as quiver244, but draws sample trajectories of the 
solution instead of the quiver plot. It will take initial conditions as (to, yo). For a single to, a 
vector of initial yo values can be passed in and the function will work correctly. This function 
can be used as 


f = O(t.y) y.*(y-5)-*(yt6) ; 


samplePlots244(f, -1, 6, -7, 6, 0, [-1,0.5,4,5], 'r') 


-1 0 1 2 3 4 5 6 


Figure A.7: Sample output from the samplePlots244 function. 


The ‘r’ here indicates that this plot will be drawn in red and put on figure 2. If this is 
combined with the quiver244 method, then it will overlay these red curves on top of the 
quiver plot drawn on figure 2. 
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function bifDiag244(f, a_min, a_max, y_min, y_max) 

This function draws a bifurcation diagram for the ode dy/dt = f(alpha, y) 
with parameter alpha running from a_min to a_maz. The axes are 
constrained to be from a_min to a_maz in the horizontal direction and 
y_min to ymax in the vertical direction. 


The black marks are for equilibrium solutions, the blue regions are where 
the solution will tend upwards, and the red region ts where it will tend 
downwards. 


HE HN HE FE FE FE NE SE 


This function will draw a bifurcation diagram for the given differential equation. Note: 
This function will need the optimization tool-box add-on for MATLAB in order to run correctly. 
As with the previous methods, it will not overwrite the figure. Example implementation: 


# = OW) Wo 2 = Blo QE 
bitDiapZa4(t i =3, 35 —b, 5, 3); 


Figure A.8: Sample output from the bi fDiag244 function. 
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function quiver2D244(f,g, X_Min, X_max, y_min, y_max, col) 
quiver2D2Z44.m 
Author: Matt Charnley 


This function draws a quiver plot for the ODE dx/dt = f(x,y), dy/dt = 
a (CG) jor 
A vcmin <= £ <= Zmax and y_min <= y <= y_maz. The functions f and g should 
= be 


f, passed in as an anonymous functions, f = @(a,y) 


The function draws this quiver plot in color col in the current figure 

and generates a normalized version (all vectors are the same length) 

as the neve frgune. SO) wnat ti can ve accessed outside Of thts funectvon. 
4, For this second figure, the magnitude of the arrows does not mean 

anything, but it ts easter to see the direction of them. 


It will start with 
hold on; and end with hold off;, so the figure needs to be cleared in the 
main file if needed. 


This function does the same concept as quiver244 but for the autonomous system of 
differential equations 


—=f(z,y) z=9(2,y). 
Example implementation: 


f @(x,y) 3.*x - 2.*x.*y; 
g @(x,y) 2.#y — 3.*x.¥*Y; 
quiver2D244(f,g, 0, 5, 0, 5, 'g'); 


function phaseLine(f, ymin, ymax) 

4 This function draws a representation of the phaseline for the 

h differential equation dy/dt = f(y). The graph is drawn from ymin to ymaz, 
4 and looks for solutions to f(y) = 0 in that region to find equilibrium 

fh solutions. This requires the Optimization Toolbox fsolve to run 
ACORReECULG. 


This function draws a representation of the phase line for an autonomous first order 
dy _ 


differential equation 4 = f(y) from Ymin tO Ymax- Example implementation: 


f = @(y) y.*(y-3).*(yt2); 


phaseLine(f, -4, 5); 
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Figure A.9: Sample output from the quiver2D244 function. 


Figure A.10: Sample output from the phaseLine function. 


ymax, tmin, tmax, x0, yO) 


function phasePortrait244(F, G, xmin, xmax, ymin, 
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The phase portrait will be draw with x bounds 


G(z,y). 


4 F(2,y) and dy/dt 


y <= ymax. It ts assumed that the initial 


A wmin <= & <= zgmax and ymin < 
% conditions £0 and yO are at £t 


0. £0 and yO 


and a sample curve 


with tmin <= 0 and tmax > 


SOE 


fA can be inputted as vectors that are the same length, 


4 will be drawn for each of them. The black dot will always be plotted at 
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This function draws a phase portrait for the two-component autonomous system a = 
F(z,y) and a = G(g,y). The axes are fixed at tmin <2 < Smac and Ymin < Y < Ymas- 
Solution curves are drawn starting at the (potential list of) points xg and yo, and will assume 
these happen at t = 0. The curves are drawn from t,,in to t,,ax, and there will be a black 


dot plotted at t,,in to indicate the direction of flow. Example implementation: 


@ S Gey) Zitse = Soe? ye 
@ = OC) —Sutke a iE 


joel PoneimreeaeMMNGE,, js, =, BG, =—8, Sp —2y Bp (tl, O, =, 1, O, =i, 
<= Elis ik gaa =i al) ¢ 


Phase Portrait 


Figure A.11: Sample output from the phasePortrait function. 


function [t, y] = rungeKuttaMethod(f, dt, Tf, TO, yO) 
4 This method solves the ODE dy/dt = f(t, y) using the Runge Kutta method 
h from t=TO to t = Tf with time step dt and initial condition yO at t = TO. 


4 In this case, f should be a function of two variables, t 
4 (time) and y. 
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function [t,y] = rungeKuttaSystemMethod(f, TO, Tf, dt, yO) 

4 This method solves the ODE system dy/dt = f(t, y) using the Runge Kutta 
~~ method 

hk from t=TO to t = Tf with time step dt and initial condition yO at t 


4 In this case, f should be a vector valued function of two variables, t 
A (time) and y (n-dimensional vector of unknowns). The length of the vector 
hk yO will determine the size of the system. 


These two methods use the Runge-Kutta method to numerically solve the differential 
equation a = f(t,y) or the system a = F(t, £). It will return the list of t and y values that 
are generated by this method. 


function [S,I,R] = SIRModel_244(r, c, ICs, Tf) 
4 This code runs an SIR model for disease spread. The system of differential 
equations used here is 
st —T#S#EL 
‘Th (ial > Clik 
TY = hdl 


h The solution ts computed using the RungeKutta method, with the helper 
4 method rungeKkuttaSystemMethod. The system is solved from t=0 to t=Tf, 
fh with initial conditions ICs given as a 3 component vector. 


function [S,1,Q,R,D] = SIRQMode1_244(alpha, beta, gamma, delta, eta, rho, 

ax IGE, We) 

4 This code runs a more complicated SIR model that adds in Q (a quarantined 
4 population) and D (a deceased population). The system of differential 
equations used here is 

Sie -alpha*S*I 

I' = alpha*S*I - (betatgammatdelta)I 

Gu beta*I - (eta + rho)Q 

R' = gamma+I + eta*l) 

ip delta*I + rhox*Q 


J 


SE SC SE SE FE SL 


4 The solution ts computed using the RungeKutta method, with the helper 
4 method rungeKkuttaSystemMethod. The system is solved from t=0 to t=Tf, 
fh with initial conditions ICs given as a 5 component vector. 
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function [S,1,Q,R,D] = SIRQVModel_244(alpha, beta, gamma, delta, eta, rho, 
4 zeta, ICs, Tf) 

4 This code runs a more complicated SIR model that adds in Q (a quarantined 
fh population) and D (a deceased population). The V component adds 

4 vaccination into the picture, where members are moved from S to R 

4 directly. The system of differential equations used here is 

si -alpha*S*I - zeta*S 

I' = alpha*S*I - (betatgammatdelta)I 

(aye beta*xI - (eta + rho)Q 

ie gamma*I + eta*Qt+zeta*s 

DY delta*I + rhox*Q 


h 
A 
h 
A 
a 
vA 
vA 


The solution ts computed using the RungeKutta method, with the helper 
method rungeKuttaSystemMethod. The system is solved from t=0 to t=Tf, 
with wnitial conditions ICs given as a 5 component vector. 


Each of these last three methods use the Runge Kutta method to numerical solve a 
disease modeling problem with their respective equations. The shared arguments are the 
initial conditions, which are a three or five component vector depending on the problem type, 
and the final time 7. The step-size used is one day, and the method will return the list of 
time-stepped values for each population (every day) from t = 0 to t = Ty. For SIR, the 
equations are 


For SIRQ, the equations are 


o =-aSI 

@ = aS! — pI -yI- 51 
= BI —~nQ- 90 

ot =a tnd 
ere 
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and for SIRQV, it is 


© =-aSI—¢8 

© =aSI~ BI -—4I 51 
= BI —nQ— 90 
al AQ 2 Co. 
<= 61+ 00 


An example implementation is 


[S,I,R] = SIRModel_244(0.1, 0.2, [0.99; 0.01; 0], 400); 
[S,1,Q,R,D] = SIRQModel_244(0.15, 0.08, 0.02, 0.03, 0.01, 0.04, [0.95; 0.05; 


as 8 @8 @l), Aoio))s 
[$,1,Q,R,D] = SIRQVModel_244(0.15, 0.08, 0.02, 0.03, 0.01, 0.04,0.2, [0.95; 
=, 0.089 O28 Os Ol, Zoo): 
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Appendix B 


Prerequisite Material 


This chapter provides a review of some of the material from previous classes that may be a 
little rusty by the time one reaches differential equations. This can be used as a reference 
whenever these topics come up throughout the book. A lot of this material (or inspiration 
for it) is taken from the Precalculus book by Stitz and Zeager [SZ]. 
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B.1 Polynomials and Factoring 


Note: Attribution: [SZ], §A.8, A.9, 2.2-2.4 


There are several components of differential equations, particularly higher order equations 
and systems, that involve dealing with and finding roots of polynomials, using these results 
to generate solutions to differential equations. This appendix will review some properties of 
and techniques related to polynomials. 


B.1.1 Definitions and Operations 


First we start with the definition of a polynomial. A polynomial is a sum of terms each 
of which is a real number or a real number multiplied by one or more variables to natural 
number powers. Some examples of polynomials are x? + xV/3 + 4, 27x?y + a and 6. Things 
; 2 De. : : : 

like 3./x, 4a — say and 13x /3y? are not polynomials. Below, we review some terminology 
about polynomials. 


Definition B.1.1 


Terms in polynomials without variables are called constant terms. 


In non-constant terms, the real number factor in the expression is called the 
coefficient of the term. 


The degree of a non-constant term is the sum of the exponents on the variables 
in the term; non-zero constant terms are defined to have degree 0. The degree of 
a polynomial is the highest degree of the nonzero terms. 


Terms in a polynomial are called like terms if they have the same variables each 
with the same corresponding exponents. 


A polynomial is said to be simplified if all arithmetic operations have been 
completed and there are no longer any like terms. 


A simplified polynomial is called a 


— monomaal if it has exactly one nonzero term 
— binomial if it has exactly two nonzero terms 


— trinomial if it has exactly three nonzero terms 


For example, 2? + 7/3 + 4 is a trinomial of degree 2. The coefficient of x? is 1 and the 
constant term is 4. The polynomial 27z?y + © is a binomial of degree 3 (ay = x*y') with 
constant term 0. 

The concept of ‘like’ terms really amounts to finding terms which can be combined using 
the Distributive Property. For example, in the polynomial 17x?y — 3xy? + 7xy?, —3xy? 
and 7xy’ are like terms, since they have the same variables with the same corresponding 
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exponents. This allows us to combine these two terms as follows: 
17x7y — 3a0y? + Try? = 172?y + (—3)cy? + Try? + 172?y + (—3 + 7)zy? = 172?y + Ary? 


Note that even though 17x7y and 4y? have the same variables, they are not like terms since 
in the first term we have x? and y = y! but in the second we have x = x! and y = y” so the 
corresponding exponents aren’t the same. Hence, 17x”y + 4xy? is the simplified form of the 
polynomial. 

There are four basic operations we can perform with polynomials: addition, subtraction, 
multiplication and division. Addition, subtraction, and multiplication follow the standard 
properties of real numbers after distributing or expanding all terms (for multiplication) and 
then collecting like terms again. Division, on the other hand, is a bit more complicated and 
will be discussed next. 


Polynomial Long Division 


We now turn our attention to polynomial long division. Dividing two polynomials follows the 
same algorithm, in principle, as dividing two natural numbers so we review that process first. 
Suppose we wished to divide 2585 by 79. The standard division tableau is given below. 


32 
79| 2585 
— 237] 
215 
—158 
“57 


In this case, 79 is called the divisor, 2585 is called the dividend, 32 is called the quotient 
and 57 is called the remainder. We can check our answer by showing: 


dividend = (divisor) (quotient) + remainder 


or in this case, 2585 = (79)(32) + 57V. We hope that the long division tableau evokes warm, 
fuzzy memories of your formative years as opposed to feelings of hopelessness and frustration. 
If you experience the latter, keep in mind that the Division Algorithm essentially is a two-step 
process, iterated over and over again. First, we guess the number of times the divisor goes 
into the dividend and then we subtract off our guess. We repeat those steps with what’s left 
over until what’s left over (the remainder) is less than what we started with (the divisor). 
That’s all there is to it! 


The division algorithm for polynomials has the same basic two steps but when we subtract 
polynomials, we must take care to subtract like terms only. As a transition to polynomial 
division, let’s write out our previous division tableau in expanded form. 
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3-10+2 
7-10+9]/2-10?+5-107+8-10+5 
—(2-10°+ 3-107+7-10) | 
2-10?+1-10+5 
—(1-10?+ 5-10 +8) 

5-10+7 


Written this way, we see that when we line up the digits we are really lining up the 
coefficients of the corresponding powers of 10 - much like how we'll have to keep the powers 
of x lined up in the same columns. The big difference between polynomial division and the 
division of natural numbers is that the value of x is an unknown quantity. So unlike using 
the known value of 10, when we subtract there can be no regrouping of coefficients as in 
our previous example. (The subtraction 215 — 158 requires us to ‘regroup’ or ‘borrow’ from 
the tens digit, then the hundreds digit.) This actually makes polynomial division easier.* 
Before we dive into examples, we first state a theorem telling us when we can divide two 
polynomials, and what to expect when we do so. 


Theorem B.1.1 (Polynomial Division) 


Let d and p be nonzero polynomials where the degree of p is greater than or equal to 
the degree of d. There exist two unique polynomials, g and r, such that p=d-q-+r, 
where either r = 0 or the degree of r is strictly less than the degree of d. 


Essentially, Theorem B.1.1 tells us that we can divide polynomials whenever the degree 
of the divisor is less than or equal to the degree of the dividend. We know we’re done with 
the division when the polynomial left over (the remainder) has a degree strictly less than the 
divisor. It’s time to walk through a few examples to refresh your memory. 


Example B.1.1: Perform the indicated division. Check your answer by showing 


dividend = (divisor) (quotient) + remainder 


1. (x3 + 4x? — 5x — 14) + (x — 2) 2. (2¢+ 7) + (3t — 4) 
3. (6y? — 1) + (2y+5) 4. (w3) + (w? — V2). 
Solution: 


1. To begin (x? + 42? — 52 — 14) + (x — 2), we divide the first term in the dividend, 
namely x°, by the first term in the divisor, namely x, and get x = x*. This then 
becomes the first term in the quotient. We proceed as in regular long division at this 
point: we multiply the entire divisor, x — 2, by this first term in the quotient to get 


*In our opinion - you can judge for yourself. 
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455 
x?(x — 2) = 2° — 2x7. We then subtract this result from the dividend. 
ga 
z—2| x® + 42? —5r—-14 
(tale). A 


6a? —5a 


Now we ‘bring down’ the next term of the quotient, namely —5x, and repeat the process 


We divide ox" = 6x, and add this to the quotient polynomial, multiply it by the divisor 
(which yields 6x(x2 — 2) = 6x? — 12x) and subtract. 


x? + 6x 
z—2| 23 +4e? — 52 —14 
~(x® 202) \ 
6x7 -— 5r = 
—(627—12r) | 
7x —14 


Finally, we ‘bring down’ the last term of the dividend, namely —14, and repeat the 


process. We divide # = 7, add this to the quotient, multiply it by the divisor (which 
yields 7(x — 2) = 7x — 14) and subtract. 


e+ 64 + 7 
a—2| 23 +47? — 52 —14 
—(x? — 227) 

6x" = 52 
—(6x? — 122) 

7x —14 

— (7x —14) 

0 


In this case, we get a quotient of x? + 62 + 7 with a remainder of 0. To check our 
answer, we compute 
(x — 2) (x? +62 +7) +0 = 2° + 62? + 7x — 22? — 122 — 14 = 2° + 40? — Ba — 14 


2. To compute (2¢ + 7) + (3t — 4), we start as before. We find # = 2, so that becomes 
the first (and only) term in the quotient. We multiply the divisor (3t — 4) by 2 and get 
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2t — 3 We subtract this from the divided and get 2 
2 


—— 
st—4| 2b + 7 
8 
—{2-—— 
3 

29 

3 


Our answer is < with a remainder of - To check our answer, we compute 


29 8 29 21 
3t—4)(2)+ 2 =2%-24+ > =2+ 5 =2t4+7V 
( (5) +5 a: 2 3 


3. When we set-up the tableau for (6y? — 1) + (2y +5), we must first issue a ‘placeholder’ 
for the ‘missing’ y-term in the dividend, 6y? — 1 = 6y? + Oy — 1. We then proceed as 
before. Since a = 3y, 3y is the first term in our quotient. We multiply (2y + 5) times 
3y and subtract it from the dividend. We bring down the —1, and repeat. 


15 

2y+5| 6y? + Oy —- 1 
—(6y*+ ly) J 
Ay = i 

75 

oe ee fees 

a) 

73 

2 


15. 13 too tS 
(2y + 5) (sy - F) + B= 0y? ~ 15y + 15y- P+ F = oy? -1¥ 


4. For our last example, we need ‘placeholders’ for both the divisor w?— /2 = w?+0w = V2 
and the dividend w? = w? + Ow? + 0w +0. The first term in the quotient is “4 = 


we 
w, and when we multiply and subtract this from the dividend, we’re left with just 
Ow? +wV2+0 = wv2. 


w2+0w—V3| w? Ow?+ Ow +0 
— (w*+0w?—wvV/2) | 
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Since the degree of wy/2 (which is 1) is less than the degree of the divisor (which is 2), 
we are done.* Our answer is w with a remainder of w/2. To check, we compute: 


(w? — V2) w+ wv? = wi — wv2+ wv? = wiv | 


B.1.2 Synthetic Division 


Usually, when we want to divide polynomials, it is because we are trying to find all roots of a 
polynomial. This comes from the idea that if we have a polynomial p(x) and a value 29 so 
that p(zo) = 0, then zo is a root of the polynomial. This means that (a — xo) is a factor of 
p(x), so that we can write 

p(x) = ( — xo)q(x) 


where q(x) is a polynomial with one lower degree than p. We can find this g(x) by dividing 


p(x) 
L— Xo’ 


q(x) = 


which is why we need division to sort this out. 

This means that we need to find the roots (or at least a root) to know what to divide 
p(x) by in order to start this process. The main theorem that can tell us where to start is 
the Rational Roots Theorem. 


Theorem B.1.2 (Rational Zeros Theorem) 


Suppose f(x) = a,@"+ap)_,0" 1+...+4,2 +4, is a polynomial of degree n with n > 1, 


and do, @1, ...@» are integers. If r is a rational zero of f, then r is of the form Ser 
where p is a factor of the constant term ay), and q is a factor of the leading coefficient 
Gas 


The Rational Zeros Theorem gives us a list of numbers to try in our synthetic division 
and that is a lot nicer than simply guessing. If none of the numbers in the list are zeros, then 
either the polynomial has no real zeros at all, or all of the real zeros are irrational numbers. 


Example B.1.2: Let f(x) = 224 + 42° — x* — 6x — 3. Use the Rational Zeros Theorem to 
list all of the possible rational zeros of f. 


Solution: To generate a complete list of rational zeros, we need to take each of the factors of 
constant term, a, = —3, and divide them by each of the factors of the leading coefficient a, = 2. 
The factors of —3 are +1 and +3. Since the Rational Zeros Theorem tacks on a + anyway, 
for the moment, we consider only the positive factors 1 and 3. The factors of 2 are 1 and 2, 
so the Rational Zeros Theorem gives the list {+ 7,£5,+2,+2} or {45,+1,43,+3}. | 


. 2 . . 
*Since Ou = 0, we could proceed, write our quotient as w + 0, and move on... but even pedants have 


limits. 
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But this still doesn’t make the process easy or straight-forward for finding the roots. 
How can we take this list of options and easily figure out where the roots are, and what the 
remaining polynomial q(x) is? 


We start by way of example: suppose we wish to determine the zeros of f(x) = 73 + 
4x? — 52 — 14. Setting f(x) = 0 results in the polynomial equation x? + 4x? — 5a — 14 = 0. 
Despite all of the factoring techniques we learned (and forgot!), this equation foils* us at 
every turn. Knowing that the zeros of f correspond to z-intercepts on the graph of y = f(z), 
we use a graphing utility to produce the graph below on the left. The graph suggests that the 
function has three zeros, one of which appears to be x = 2 and two others for whom we are 
provided what we assume to be decimal approximations: « + —4.414 and x + —1.586. We 
can verify if these are zeros easily enough. We find f(2) = (2)? + 4(2)? — 5(2) — 14 = 0, but 
f(—4.414) = 0.0039 and f(—1.586) = 0.0022, While these last two values are probably by 
some measures, ‘close’ to 0, they are not exactly equal to 0. The question becomes: is there a 
way to use the fact that x = 2 is a zero to obtain the other two zeros? Based on our experience, 
if x = 2 is a zero, it seems that there should be a factor of (a — 2) lurking around in the 
factorization of f(x). In other words, we should expect that x2? + 4x? — 5x —14 = (x — 2) q(z), 
where q(x) is some other polynomial. How could we find such a q(x), if it even exists? The 
answer comes from our old friend, polynomial division. Below on the right, we perform the 
long division: (x? + 4x? — 5a — 14) + (x — 2) and obtain x? + 62x + 7. 


x? + 62+ 7 
x—2| 23 +47? — 5a —14 
—(x? — 227) 

6x27 — br 
—(6x? — 122) 

7x —14 

— (7x —14) 

0 


Said differently, f(x) = 23 + 4a? — 5x2 — 14 = (x — 2) (x? +6x +7). Using this form of 
f(x), we find the zeros by solving (x — 2) (x? + 6r +7) = 0. Setting each factor equal to 
0, we get x — 2 = 0 (which gives us our known zero, x = 2) as well as v7 + 624+7 = 0. 
The latter doesn’t factor nicely, so we apply the Quadratic Formula to get x = —3 + V2. 
Sure enough, —3 — J/2 x —4.414 and —3 + V2 & —1.586. We leave it to the reader to show 
f(—3 — /2) =0 and f(-34+ V2) =0. 


The point of this section is to generalize the technique applied here. First up is a friendly 
reminder of what we can expect when we divide polynomials. 


*pun intended 


B.1. POLYNOMIALS AND FACTORING 459 


Theorem B.1.3 


Suppose d(x) and p(x) are nonzero polynomial functions where the degree of p is greater 


than or equal to the degree of d. There exist two unique polynomial functions, q(x) 
and r(x), such that p(x) = d(x) q(x) + r(x), where either r(x) = 0 or the degree of r is 
strictly less than the degree of d. 


As you may recall, all of the polynomials in Theorem B.1.3 have special names. The 
polynomial p is called the dividend; d is the divisor; q is the quotient; r is the remainder. If 
r(az) = 0 then d is called a factor of p. The word ‘unique’ here is critical in that it guarantees 
there is only one quotient and remainder for each division problem.* The proof of Theorem 
B.1.3 is usually relegated to a course in Abstract Algebra, but we can still use the result to 
move forward with the rest of this section. 

If we want to find all of the roots of a polynomial in a reasonable way, we had better find a 
more efficient way to divide polynomial functions by quantities of the form « —c. Fortunately, 
people like Ruffini and Horner have already blazed this trail. Let’s take a closer look at the 
long division we performed at the beginning of the section and try to streamline it. First off, 
let’s change all of the subtractions into additions by distributing through the —1s. 


x? + 6x +7 
e=9) 2 dg? = be —14 
=? 4 Oy? 
62? — 5a 
—627+4+ 122 
7x —14 
—7x+14 
0 


Next, observe that the terms —x?, —6x? and —7z are the exact opposite of the terms 
above them. The algorithm we use ensures this is always the case, so we can omit them 
without losing any information. Also note that the terms we ‘bring down’ (namely the —5x 
and —14) aren’t really necessary to recopy, so we omit them, too. 


z?+62+7 
e=9)| ¢? 4-477 = be —14 
2x? 
6x? 
12% 
12 
14 
0 


Let’s move terms up a bit and copy the z° into the last row. 


*Hence the use of the definite article ‘the’ when speaking of the quotient and the remainder. 
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ve+6r2+7 
e—2| 7°9+4n?—-— 5x2 -14 
Qn? 127 14 


eo Gee “Ta: G 
Note that by arranging things in this manner, each term in the last row is obtained by 
adding the two terms above it. Notice also that the quotient polynomial can be obtained by 
dividing each of the first three terms in the last row by x and adding the results. If you take 
the time to work back through the original division problem, you will find that this is exactly 
the way we determined the quotient polynomial. This means that we no longer need to write 
the quotient polynomial down, nor the x in the divisor, to determine our answer. 


—2| 2? +42?— 52 —14 

Qe? 12e° 14 

e662" 7a. 0 
We’ve streamlined things quite a bit so far, but we can still do more. Let’s take a moment 
to remind ourselves where the 2x”, 12% and 14 came from in the second row. Each of these 
terms was obtained by multiplying the terms in the quotient, x”, 6x and 7, respectively, by 
the —2 in x — 2, then by —1 when we changed the subtraction to addition. Multiplying by 
—2 then by —1 is the same as multiplying by 2, so we replace the —2 in the divisor by 2. 
Furthermore, the coefficients of the quotient polynomial match the coefficients of the first 
three terms in the last row, so we now take the plunge and write only the coefficients of the 

terms to get 


2| 14 -5 -14 
Ao. 1A 
lige - <0 


We have constructed a synthetic division tableau for this polynomial division problem. 
Let’s re-work our division problem using this tableau to see how it greatly streamlines the 
division process. To divide x? + 4x? — 5a — 14 by x — 2, we write 2 in the place of the divisor 
and the coefficients of 2° + 4x? — 5x2 — 14 in for the dividend. Then ‘bring down’ the first 
coefficient of the dividend. 


2) 14 -5 -14 2) 14 -5 14 
| 
1 


Next, take the 2 from the divisor and multiply by the 1 that was ‘brought down’ to get 2. 
Write this underneath the 4, then add to get 6. 
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Now take the 2 from the divisor times the 6 to get 12, and add it to the —5 to get 7. 


2) 14 -5 14 2) 14 -5 -14 
5 2 32 Lo 
1 6 16 7 


Finally, take the 2 in the divisor times the 7 to get 14, and add it to the —14 to get 0. 


2) 14 -5 —14 2) 14 -5 -14 
heiB 1a, a 1 2 12 14 
16 7 16 7 {0 


The first three numbers in the last row of our tableau are the coefficients of the quotient 
polynomial. Remember, we started with a third degree polynomial and divided by a first 
degree polynomial, so the quotient is a second degree polynomial. Hence the quotient is 
x? +6x+7. The number in the box is the remainder. Synthetic division is our tool of choice 
for dividing polynomials by divisors of the form x — c. It is important to note that it works 
only for these kinds of divisors.* Also take note that when a polynomial (of degree at least 1) 
is divided by x — c, the result will be a polynomial of exactly one less degree. Finally, it is 
worth the time to trace each step in synthetic division back to its corresponding step in long 
division. While the authors have done their best to indicate where the algorithm comes from, 
there is no substitute for working through it yourself. 


Example B.1.3: Use synthetic division to perform the following polynomial divisions. Iden- 
tify the quotient and remainder. 


3 O2 sits pps a A — 8z — 122? 
1. (5a° — 2a* + 1)+(x—-3) 2. (8 +8) + (t+2) 3 * 4 z 
= 


Solution: 


1. When setting up the synthetic division tableau, the coefficients of even ‘missing’ terms 
need to be accounted for, so we enter 0 for the coefficient of x in the dividend. 


3| 5 -2 0 1 
| 45-39: 117 


5 13 39 {118 


Since the dividend was a third degree polynomial function, the quotient is a second degree 
(quadratic) polynomial function with coefficients 5, 13 and 39: q(x) = 5x2? + 13x 4+ 39. 
The remainder is r(x) = 118. According to Theorem B.1.3, we have 5x? — 277 +1 = 
(x — 3) (54? + 132 + 39) + 118, which we leave to the reader to check. 


*You’ll need to use good old-fashioned polynomial long division for divisors of degree larger than 1. 
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2. To use synthetic division here, we rewrite t + 2 as t — (—2) and proceed as before 


-2)1 00 8 
| 2 4 -8 
1 -2 4 [0] 
We get the quotient q(t) = t? —2t+4 and the remainder r(t) = 0. Relating the dividend, 


quotient and remainder gives: t? + 8 = (t + 2) (t? — 2t +4), which is a specific instance 
of the ‘sum of cubes’ formula some of you may recall. 


3. To divide 4 — 8z — 122? by 2z — 3, two things must be done. First, we write the 
dividend in descending powers of z as —12z? — 8z + 4. Second, since synthetic division 
works only for factors of the form z — c, we factor 2z — 3 as 2 (2 — 3). Hence, we are 
dividing —12z? — 8z + 4 by two factors: 2 and (z — 2). Dividing first by 2, we obtain 


—62z" — 4z +2. Next, we divide —6z? — 4z + 2 by (z — 3): 


3| -6 -4 
1-9 -¥% 
—~§6 —13 |—35 


2 


Hence, —627 —4z+2= (z = 3) (—6z — 13) - 2 However when it comes to writing 
the dividend, quotient and remainder in the form given in Theorem B.1.3, we need to 
find q(z) and r(z) so that —12z* — 8z4 4 = (2z — 3)q(z) + r(z). Hence, starting with 
—627 — 4z +2 = (z — 3) (—6z — 13) — 82, we multiply 2 back on both sides: 


6284242 = (2-3) (-62-18)-4 
2(-622 -42+2) = 2[(z- 3) (-62-18) - 2] 
—12z7-8z+4 = 2(z 3 


—1227-8z+4 = (2z—3)(—6z — 13) —35 


At this stage, we have written —12z? — 8z + 4 in the form (2z — 3)q(z) + r(z), so we 
identify the quotient as q(z) = —6z — 13 and the remainder is r(z) = —35. But how 
can we be sure these are the same quotient and remainder polynomial functions we 
would have obtained if we had taken the time to do the long division in the first place? 
Because of the word ‘unique’ in Theorem B.1.3. The theorem states that there is only 
one way to decompose —122? — 8z + 4 as (22 — 3)q(z) + r(z). Since we have found such 
a way, we can be sure it is the only way.* 2 


The next example pulls together all of the concepts discussed in this section. 


Example B.1.4: Let p(x) = 2x3 — 52 4+ 3. 


1. Find p(—2) using The Remainder Theorem. Check your answer by substitution. 


*But it wouldn’t hurt to check, just this once. 
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2. Verify x = 1 is a zero of p and use this information to all the real zeros of p. 
Solution: 


1. The Remainder Theorem states p(—2) is the remainder when p(x) is divided by x—(—2). 
We set up our synthetic division tableau below. We are careful to record the coefficient 


of z* as 0: 
-2;2 0-5 3 
{ -4 8 -6 
2-4 3 |-3 
According to the Remainder Theorem, p(—2) = —3. We can check this by direct 


substitution into the formula for p(x): p(—2) = 2(—2)? —5(—2) + 3 = -16+10+3 = —3. 


2. We verify x = 1 is a zero of p by evaluating p(1) = 2(1)? — 5(1) +3 = 0. To see if there 
are any more real zeros, we need to solve p(x) = 2x? — 52 +3 = 0. From the Factor 
Theorem, we know since p(1) = 0, we can factor p(x) as (x — 1)q(x). To find g(x), we 
use synthetic division: 


1120-5 8 
oR 08 


2 2 -3 |0| 


As promised, our remainder is 0, and we get p(x) = (a — 1) (2x7 + 2x — 3). Setting 
this form of p(x) equal to 0 we get (x — 1) (2x? + 2x — 3) = 0. We recover x = 1 from 
setting x — 1 = 0 but we also obtain x = stave from 2x? + 27 — 3 = 0, courtesy of the 


Quadratic Formula. 
eal 


Our next example demonstrates how we can extend the synthetic division tableau to 
accommodate zeros of multiplicity greater than 1. 


Example B.1.5: Let p(x) = 424 — 423 — 11x? + 12x — 3. Show x = is a zero of multiplicity 
2 and find all of the remaining real zeros of p. 


Solution: While computing p (4) = 0 shows x = 5 is a zero of p, to prove it has multiplicity 
2, we need to factor p(x) = (a — 1)? q(x) with q (4) #0,. We set up for synthetic division, 
but instead of stopping after the first division, we continue the tableau downwards and divide 
(x - 5) directly into the quotient we obtained from the first division as follows: 


5| 4 -4 -ll 12 -3 
L 2 -1 -6 3 
| 4 -2 -12 6 [0] 
2 0 -6 
4 0 —12 |0| 
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We get:* 4a* — 473 — 112? + 122-3 = (a — ae (4x? — 12). Note if we let q(x) = 427-12, 
then q (4) =A4 Gy — 12 = —11 £0 which proves x = 5 is a zero of p of multiplicity 2. To 
find the remaining zeros of p, we set the quotient 47? — 12 = 0, so x? = 3 and extract square 
roots to get x = +3. | 


One last wrinkle in this process is complex roots, since it is possible for a polynomial 
(particularly a quadratic polynomial) to have complex numbers as roots. For a reminder of 
some more properties of complex numbers see § B.2. For this section in particular, we only 
need a few basic facts. 


For us, it suffices to review the basic vocabulary. 
Definition B.1.2 


e The imaginary unit 2 = /—1 satisfies the two following properties 


ie il 


2. If cis areal number with c > 0 then /—c = ivV/c 


e The complex numbers are the set of numbers C = {a+ bi|a,b € R} 


e Given a complex number z = a+bi, the complex conjugate of z, Z = a+ bi 


Note that every real number is a complex number, that is R C C. To see this, take your 
favorite real number, say 117. We may write 117 = 117 + 02 which puts in the form a+ bi. 
Hence, we we speak of the ‘complex zeros’ of a polynomial function, we are talking about not 
just the non-real, but also the real zeros. 


Complex numbers, by their very definition, are two dimensional creatures. To see this, we 
may identify a complex number z = a+ bi with the point in the Cartesian plane (a,b). The 
horizontal axis is called the ‘real’ axis since points here have the form (a, 0) which corresponds 
to numbers of the form z = a+ 0i = a which are the real numbers. The vertical axis is called 
the ‘imaginary’ axis since points here are of the form (0,b) which correspond to numbers 
of the form z = 0 + bi = 01, the so-called ‘purely imaginary’ numbers. Below we plot some 
complex numbers on this so-called ‘Complex Plane.’ Plotting a set of complex numbers this 
way is called an Argand Diagram, and opens up a wealth of opportunities to explore many 
algebraic properties of complex numbers geometrically. For example, complex conjugation 
amounts to a reflection about the real axis, and multiplication by 7 amounts to a 90° rotation. 
While we won’t have much use for the Complex Plane in this section, it is worth introducing 
this concept now, if, for no other reason, it gives the reader a sense of the vastness of the 
complex number system and the role of the real numbers in it. 


Returning to zeros of polynomials, suppose we wish to find the zeros of f(x) = x? — 2x +5. 
To solve the equation 7? — 2x + 5 = 0, we note that the quadratic doesn’t factor nicely, so we 


*For those wanting more detail: the first division gives: 4a4 Ax? lla? + 122 30 
(a — 4) (4x3 — 2a? — 12a + 6). The second division gives: 4° — 2a? — 12a + 6 = (a — 4) (4a? — 12). 
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resort to the Quadratic Formula and obtain 


_ -(-2)4 J 22-406) _ 24 V=16 = 2 +4 
- (1) = 


= leo 


Two things are important to note. First, the zeros 1 + 27 and 1 — 22 are complex conjugates. 
If ever we obtain non-real zeros to a quadratic function with real number coefficients, the 
zeros will be a complex conjugate pair. (Do you see why?) 

We could ask if all of the theory of polynomial division holds for non-real zeros, in 
particular the division algorithm and the Remainder and Factor Theorems. The answer is 


6 P} 


yes. 


1+2i| 1 —2 5 
} 142) =5 
1 -1+2i [0] 


Indeed, the above shows 2? — 2x +5 = (a —[1+4 2%])(a—1+ 27) = (a — [1 +2i%])(x — [1 — 27]) 
which demonstrates both (x — [1 + 27]) and (x — [1 — 2i]) are factors of x? — 27 + 5.* 

But how do we know if a general polynomial has any complex zeros at all? We have 
many examples of polynomials with no real zeros. Can there be polynomials with no zeros 
whatsoever? The answer to that last question is “No.” and the theorem which provides that 
answer is The Fundamental Theorem of Algebra. 


Theorem B.1.4 (The Fundamental Theorem of Algebra) 


Suppose f is a polynomial function with complex number coefficients of degree n > 1, 
then f has at least one complex zero. 


The Fundamental Theorem of Algebra is an example of an ‘existence’ theorem in Math- 
ematics. Like the Intermediate Value Theorem, the Fundamental Theorem of Algebra 
guarantees the existence of at least one zero, but gives us no algorithm to use in finding it. 
In fact, as we mentioned previously, there are polynomials whose real zeros, though they 
exist, cannot be expressed using the ‘usual’ combinations of arithmetic symbols, and must be 
approximated. It took mathematicians literally hundreds of years to prove the theorem in 
its full generality,’ and some of that history is recorded . Note that the Fundamental 
Theorem of Algebra applies to not only polynomial functions with real coefficients, but to 
those with complex number coefficients as well. 

Suppose f is a polynomial function of degree n > 1. The Fundamental Theorem of 
Algebra guarantees us at least one complex zero, z,. The Factor Theorem guarantees that 
f(x) factors as f(a) = (x — z,) q(x) for a polynomial function qg,, which has degree n — 1. 
If n —1 > 1, then the Fundamental Theorem of Algebra guarantees a complex zero of 
q, as well, say z, so then the Factor Theorem gives us q,(@) = (a — 22) q(x), and hence 
f(a) = (a — 2,) (a — 2) q(x). We can continue this process exactly n times, at which point 


*It is a good review of the algebra of complex numbers to start with (a — [1 + 2i])(a — [1 — 2¢]), perform 
the indicated operations, and simplify the result to x? — 2x +5. See part 6 of Example B.2.1. 
‘So if its profound nature and beautiful subtlety escape you, no worries! 
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our quotient polynomial gq, has degree 0 so it’s a constant. This constant is none-other than 
the leading coefficient of f which is carried down line by line each time we divide by factors 
of the form x — c. 


Theorem B.1.5 (Complex Factorization Theorem) 


Suppose f is a polynomial function with complex number coefficients. If the degree 
of f isn and n > 1, then f has exactly n complex zeros, counting multiplicity. If z,, 
Za, ..-, 2%, are the distinct zeros of f, with multiplicities m,, m., ..., m,, respectively, 
then f(x) = a(x — z,)"™" (4 — 2)" ---(@— %)". 


Theorem B.1.5 says two important things: first, every polynomial is a product of linear 
factors; second, every polynomial function is completely determined by its zeros, their 
multiplicities, and its leading coefficient. We put this theorem to good use in the next 
example. 


Example B.1.6: Let f(x) = 122° — 20z* + 192° — 6x? — 27 +1. 
1. Find all of the complex zeros of f and state their multiplicities. 


2. Factor f(x) using Theorem B.1.5 
Solution: 


1. Since f is a fifth degree polynomial, we know that we need to perform at least three 
successful divisions to get the quotient down to a quadratic function. At that point, 
we can find the remaining zeros using the Quadratic Formula, if necessary. Using the 
techniques of synthetic division: 


+| 12 -20 19 -6 -2 1 
1 6 -7 6 O -1 

‘| 12 -14 12 0 -2 [0] 
1 6-4 4 2 

-1/12 -8 8 4 (0 


| -4 4 -4 
12 -12 12 |0| 


Our quotient is 127%? — 127 + 12, whose zeros we find to be ies From Theorem B.1.5, 


we know f has exactly 5 zeros, counting multiplicities, and as such we have the zero 5 
with multiplicity 2, and the zeros —3, enue and —, each of multiplicity 1. 


2. Applying Theorem B.1.5, we are guaranteed that f factors as 


fey =12 (2-3) (2+2) (+ ) («= [-4)) 


1+iV3 
9 
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= 


A true test of Theorem B.1.5 would be to take the factored form of f(x) in the previous 
example and multiply it out* to see that it really does reduce to f(x) = 12x° — 20x4 + 19x? — 
6x? — 2x + 1. When factoring a polynomial using Theorem B.1.5, we say that it is factored 
completely over the complex numbers, meaning that it is impossible to factor the polynomial 
any further using complex numbers. If we wanted to completely factor f(x) over the real 
numbers then we would have stopped short of finding the nonreal zeros of f and factored f 
using our work from the synthetic division to write f(x) = (a — 4) ‘ (a + §) (120? — 122 + 12), 


or f(x) = 12 (x — He (x + 4) (x? —x +1). Since the zeros of x? — x + 1 are nonreal, we call 
x? —a +1 an irreducible quadratic meaning it is impossible to break it down any further 
using real numbers. 

The last two results of the section show us that, theoretically, the non-real zeros of poly- 
nomial functions with real number coefficients come exclusively from irreducible quadratics. 


Theorem B.1.6 (Conjugate Pairs Theorem) 


If f is a polynomial function with real number coefficients and z is a complex zero of 
f, then so is Z. 


To prove the theorem, let f(x) = an@"+@n_,0" '+...+4@,%? + a,2+4+ a) be a polynomial 
function with real number coefficients. If z is a zero of f, then f(z) = 0, which means 
Anz” + An12™ 1 +...+422? +a,2 +a, = 0. Next, we consider f (Z) and apply Theorem B.2.1 
below. 


f(2Z) = an(Z)"+ an. (Zz) +... +a, (Z) +a,2%4+ ay 


= AnZ™ + An 12" +... + 2+ Zt A since (2)" = 2" 
= Gn2"+G,22" '4+...4624+074+0 since the coefficients are real 
= On2"” + a,_12" 1+...+4.27+0274+0 since ZW = ZW 
= Anz” + An-12""* +... + 227 + 4,2 + Ay since Z+w=z+w 
= f(z) 

= 

= 0 


This shows that Z is a zero of f. So, if f is a polynomial function with real number 
coefficients, Theorem B.1.6 tells us that if a + bi is a nonreal zero of f, then so is a — bi. In 
other words, nonreal zeros of f come in conjugate pairs. The Factor Theorem kicks in to give 
us both («—|a+bi]) and (x—|a—bi]) as factors of f(x) which means (a —|a+bi])(2—[a—bi]) = 
x? + 2ax + (a? + b?) is an irreducible quadratic factor of f. As a result, we have our last 
theorem of the section. 


*This is a good chance to test your algebraic mettle and see that all of this does actually work. 
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Theorem B.1.7 (Real Factorization Theorem) 


Suppose f is a polynomial function with real number coefficients. Then f(x) can 
be factored into a product of linear factors corresponding to the real zeros of f and 
irreducible quadratic factors which give the nonreal zeros of f. 


Example B.1.7: Let f(x) = «* + 64. 
1. Use synthetic division to show that 7 = 2+ 22 is a zero of f. 
2. Find the remaining complex zeros of f. 
3. Completely factor f(x) over the complex numbers. 
4. Completely factor f(x) over the real numbers. 
Solution: 


1. Remembering to insert the 0’s in the synthetic division tableau we have 


2+2:|1 O 0 0 64 
1 242i 8 -16+16i —64 
1 24+2i 8 -—16+16i |0| 


2. Since f is a fourth degree polynomial, we need to make two successful divisions to get 
a quadratic quotient. Since 2 + 27 is a zero, we know from Theorem B.1.6 that 2 — 27 is 
also a zero. We continue our synthetic division tableau. 


2+2i| 1 0 0 0 64 
L 24+2i 81 -16+16i —64 

2-2i| 1 2+2 8  -16+16i [0] 
| 2-21 8-81 16-16: 


1 4 8 10] 


Our quotient polynomial is 7? + 4x + 8. Using the quadratic formula, we solve 
x? + 4x +8 = 0 and find the remaining zeros are —2 + 2i and —2 — 2i. 


3. Using Theorem B.1.5, we get f(x) = (x—[2—23])(a2—[2422]) (a —[—2+ 2i]) (a2 —[—2—2i]). 


4. To find the irreducible quadratic factors of f(x), we multiply the factors together which 
correspond to the conjugate pairs. We find (x — [2 — 27])(x — [2 + 27]) = a2? — 47 + 8, 
and (a — [—2 + 27])(a — [—2 — 2i]) = 7? +47 +8, so f(x) = (a? — 4x + 8) (x? + 42 7) 


We close this section with an example where we are asked to manufacture a polynomial 
function with certain characteristics. 


Example B.1.8: Find a polynomial function p of lowest degree that has integer coefficients 
and satisfies all of the following criteria: 
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e the graph of y = p(x) touches and rebounds from the z-axis at (3, 0) 
e x = 31 is a zero Of p. 
e as © — —o0, p(x) 4 —oo 


e as © — 00, p(x)  —oo 


Solution: 

To solve this problem, we will need a good understanding of the relationship between 
the x-intercepts of the graph of a function and the zeros of a function, the Factor Theorem, 
the role of multiplicity, complex conjugates, the Complex Factorization Theorem, and end 
behavior of polynomial functions. (In short, you’ll need most of the major concepts of this 
chapter.) Since the graph of p touches the z-axis at (5,0), we know © = 5 is a zero of 
even multiplicity. Since we are after a polynomial of lowest degree, we need x = ; to have 


multiplicity exactly 2. The Factor Theorem now tells us (a — ay is a factor of p(x). Since 
x = 3% is a zero and our final answer is to have integer (hence, real) coefficients, x = —3:2 is 
also a zero. The Factor Theorem kicks in again to give us (x — 32) and (x + 32) as factors of 
p(x). We are given no further information about zeros or intercepts so we conclude, by the 
Complex Factorization Theorem that p(x) = a (a — Ay (x — 3i)(a + 37) for some real number 
a. Expanding this, we get p(x) = ax* — 2423 + %4x? — 6ax +a. In order to obtain integer 
coefficients, we know a must be an integer multiple of 9. Our last concern is end behavior. 
Since the leading term of p(x) is ax‘, we need a < 0 to get p(x) + —o0 as x — too. Hence, 
if we choose x = —9, we get p(x) = —9x*+ 6x? — 82x? +542 —9. We can verify our handiwork 


using the techniques developed in this chapter. _| 
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B.2 Complex Numbers 


Note: Attribution: [SZ], §A.11 


The equation x? + 1 = 0 has no real number solutions. However, it would have solutions if 
we could make sense of /—1. The Complex Numbers do just that - they give us a mechanism 
for working with /—1. As such, the set of complex numbers fill in an algebraic gap left by 
the set of real numbers. 

Here’s the basic plan. There is no real number x with x? = —1, since for any real number 
x? > 0. However, we could formally extract square roots and write x = +,/—1. We build 
the complex numbers by relabeling the quantity /—1 as i, the unfortunately misnamed 
imaginary unit.* The number 7, while not a real number, is defined so that it plays along 
well with real numbers and acts very much like any other radical expression. For instance, 
3(27) = 61, 7i — 37 = 47, (2 — 71) + (8+ 47) = 5 — 31, and so forth. The key properties which 
distinguish 7 from the real numbers are listed below. 


Definition B.2.1 


The imaginary unit 2 satisfies the two following properties: 


el 


2. If cis a real number with c > 0 then /—c = ivV/c 


Property 1 in the previous definition establishes that 7 does act as a square root! of —1, 
and property 2 establishes what we mean by the ‘principal square root’ of a negative real 
number. In property 2, it is important to remember the restriction on c. For example, it is 
perfectly acceptable to say //—4 = iV4 = i(2) = 2i. However, \/—(—4) 4 iV/—4, otherwise, 
we'd get 


2=V74=4/-(—4 =i/—4= 121) = 277 = 2(—1) = -2, 


which is unacceptable. The moral of this story is that the general properties of radicals do 
not apply for even roots of negative quantities. With Definition B.2.1 in place, we can define 
the set of complex numbers. 

A complex number is a number of the form a + bi, where a and 0 are real numbers and i 
is the imaginary unit. The set of complex numbers is denoted C. 


Complex numbers include things you’d normally expect, like 3+ 22 and 2 — iv/3. However, 
don’t forget that a or b could be zero, which means numbers like 37 and 6 are also complex 
numbers. In other words, don’t forget that the complex numbers include the real numbers,* 
so 0 and x — V21 are both considered complex numbers. The arithmetic of complex numbers 


*Some Technical Mathematics textbooks label it ‘j’. While it carries the adjective ‘imaginary’, these 
numbers have essential real-world implications. For example, every electronic device owes its existence to the 
study of ‘imaginary’ numbers. 

‘Note the use of the indefinite article ‘a’. Whatever beast is chosen to be i, —i is the other square root of 
—1. 

In the language of set notation, R C C. 
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is as you would expect. The only things you need to remember are the two properties above. 
The next example should help recall how these animals behave. 


Example B.2.1: Perform the indicated operations. 


1. =o) =a) 2. (1 — 2%)(3 + 4i) — 
aN sy | 5. 4) (812) 6. (x—[1+2%])(a— [1 —24)) 
Solution: 


1. As mentioned earlier, we treat expressions involving 7 as we would any other radical. 
We distribute and combine like terms: 


(1—21)-(8+41) = 1-2%-3-4 Distribute 
= —2-—6i Gather like terms 


Technically, we’d have to rewrite our answer —2 — 67 as (—2) + (—6)i to be (in the 
strictest sense) ‘in the form a+ bi’. That being said, even pedants have their limits, so 
—2 — 67 is good enough. 


2. Using the Distributive Property (a.k.a. F.O.1.L.), we get 


(1—2i)(3 44%) = (1)(3) + (1)(42) — (28)(3) — (21)(48) F.O.LL. 


= 3441-61 — 8? 


= 3—2)—8(-1) ?@=-1 
= 3-248 
= 11-2i 


3. How in the world are we supposed to simplify cared Well, we deal with the denominator 


3 — 47 as we would any other denominator containing two terms, one of which is a 
square root. We multiply both numerator and denominator by 3 + 47, the (complex) 
conjugate of 3 — 42. Doing so produces 


1— 2% (1 — 24)(3 + 4i) 
= Equivalent Fract 
3_ Aj (3 _ di) (3 ie di) qulvalen ract1ons 


3+ 4i — 6i — 87? 


= F.O.LL. 
9 — 1672 
39) — 9(— 
_ i — 8(-1) 2@o-1 
9 — 16(—1) 
_ i-2i 
i 25 
it 2. 
= on os 


4. We use property 2 of Definition B.2.1 first, then apply the rules of radicals applicable 
to real numbers to get /—3,/—12 = (iV3) (iV12) = 77/3 -12 = —\/36 = —6. 
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5. We adhere to the order of operations here and perform the multiplication before the 
radical to get \/(—3)(—12) = V36 = 6. 


6. We brute force multiply using the distributive property and find that 


(a — [1 + 2%])(a — [1 — 27]) x? — x{1 — 27] — x[1 + 27] + [1 — 2][1 + 27] 
= g?—2¢42ie —2 —Qin+1-—214+ 21 — 4: 
x? — 2x +1 -4(-1) 


= g*-2Qr+5 


| 


In the previous example, we used the ‘conjugate’ idea from simplifying radical equations 
to divide two complex numbers. More generally, the complex conjugate of a complex number 
a+ bi is the number a — bi. The notation commonly used for complex conjugation is a ‘bar’: 
a+ bi =a-— bi. For example, 3 + 22 = 3 — 2i and 3 — 22 = 3+ 2i. To find 6, we note that 
6=6+0i = 6—0: = 6, so 6 = 6. Similarly, 42 = —4i, since 42 = 0 + 42 = 0— 47 = —4i. Note 
that 3+ V5 = 34+ V5, not 3— V5, since 3+ V5 =34+ V54 0i = 34+ V5—0i =34+V5. Here, 
the conjugation specified by the ‘bar’ notation involves reversing the sign before i = /—1, 
not before 5. The properties of the conjugate are summarized in the following theorem. 


Theorem B.2.1 (Properties of the Complex Conjugate) 


Let z and w be complex numbers. 


e 2” = (Z)”, for any natural number n 


e z is areal number if and only if Z = z. 


Theorem B.2.1 says in part that complex conjugation works well with addition, multi- 
plication and powers. The proofs of these properties can best be achieved by writing out 
z=a+t+bi and w =c+di for real numbers a, b, c and d. Next, we compute the left and right 
sides of each equation and verify that they are the same. 

The proof of the first property is a very quick exercise.* To prove the second property, we 
compare z+ w with 7+ Ww. We have 7+W=a+bit+ct+di=a-—bi+c-—di. To find z+ wu, 
we first compute 


z+w = (a+ bi) + (c+di) = (a+c)+(b+d)i 


sO 


z+w=(at+o)+(b4+ di =(at+c)—(b+dji=a+c—bi-di=a-—bi+c-di=74+U 


*Trust us on this. 
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As such, we have established z + w = Z+W. The proof for multiplication works similarly. The 
proof that the conjugate works well with powers can be viewed as a repeated application of 
the product rule, and is best proved using a technique called Mathematical Induction.The last 
property is a characterization of real numbers. If z is real, then z = a+0i, soz = a—0i =a = z. 
On the other hand, if z = Z, then a+ bi = a — bi which means b = —b so b = 0. Hence, 
z=a-+0i =a and is real. 

We now return to the business of solving quadratic equations. Consider 7? — 27 + 5 = 0. 
The discriminant b? — 4ac = —16 is negative, so we know that there are no real solutions, 
since the Quadratic Formula would involve the term /—16. Complex numbers, however, are 
built just for such situations, so we can go ahead and apply the Quadratic Formula to get: 


_ -(-2) + J/H2 =40)(6) _ 2+ V=16 = 2 + 4s 
_ 2(1) a or 


Example B.2.2: Find the complex solutions to the following equations.* 


= se 2. 


er 2. 2t4 = 917 +5 3. 2+1=0 


Solution: 


1. Clearing fractions yields a quadratic equation so we then proceed via normal quadratic 
equation methods. 


22 
= £+3 
e+l1 
2x = (#+3)(a+1) Multiply by (x + 1) to clear denominators 
Qe = #?+24+324+3 F.O.LL. 
Qa = 27+474+3 Gather like terms 
(= oF a7 +9 Subtract 2x 


From here, we apply the Quadratic Formula 


~2+ ,/2? — 4(1)(3) 


a a1) Quadratic Formula 
ee ae 
= — Simplify 
Laas 
= = Definition of 7 
—2+12V/2 
= a Product Rule for Radicals 


2(-1 + iv2) 


= 7 Factor and reduce 
= -1+iV2 


We get two answers: « = —1+i,/2 and its conjugate x = —1 —i/2. Checking both of 
these answers reviews all of the salient points about complex number arithmetic and is 
therefore strongly encouraged. 


*Remember, all real numbers are complex numbers, so ‘complex solutions’ means both real and non-real 
answers. 


ATA 
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2. Since we have three terms, and the exponent on one term (‘4’ on t*) is exactly twice 


the exponent on the other (‘2’ on t?), we have a Quadratic in Disguise. We proceed 
accordingly. 


24 = 9+4+5 
2° = 9F =5 = 0 Subtract 9t? and 5 
(2t7+1)(t?-5) = 0 Factor 


2#7+1=0 or #?=5 Zero Product Property 


From 2t? + 1 = 0 we get 2t? = —1, or t? = —}. We extract square roots as follows: 


H i 1 1 1/2 
(=5y) 2 = tif} =i) aay ore) 
2 2 a 2 J2 2 


where we have rationalized the denominator per convention. From ¢? = 5, we get 
t= +4+¥V5. In total, we have four complex solutions - two real: t = +,/5 and two 
non-real: t = iv 


. To find the real solutions to z? + 1 = 0, we can subtract the 1 from both sides and 


extract cube roots: z? = —1, so z = </—1 = —1. It turns out there are two more 
non-real complex number solutions to this equation. To get at these, we factor: 


2+1 = 0 
(z+1)(22-2z+1) = 0 Factor (Sum of Two Cubes) 
z+1=0 or 227-z+1=0 


From z+ 1 = 0, we get our real solution z = —1. From z? — z + 1 = 0, we apply the 
Quadratic Formula to get: 


_—(-)+ VOIP ADD _ 14 v=3 _ 1 tiv3 


o— 


2(1) 2 2 
Thus we get three solutions to z?+ 1 = 0 - one real: z = —1 and two non-real: 
= Lis As always, the reader is encouraged to test their algebraic mettle and check 


these solutions. 


= 


It is no coincidence that the non-real solutions to the equations in Example B.2.2 appear 


in complex conjugate pairs. Any time we use the Quadratic Formula to solve an equation 
with real coefficients, the answers will form a complex conjugate pair owing to the + in the 
Quadratic Formula. 
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Theorem B.2.2 (Discriminant Theorem) 


Given a Quadratic Equation ax? + bx + c = 0, where a, b and © are real numbers, let 
D = b? — 4ac be the discriminant. 


e If D > 0, there are two distinct real number solutions to the equation. 


e If D =0, there is one (repeated) real number solution. 


‘Repeated’ here comes from the fact that ‘both’ solutions —e0 reduce to —2. 


e If D <0, there are two non-real solutions which form a complex conjugate pair. 
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B.3 Differentiation and Integration Techniques 


In this section, we will cover some of the basic derivative and integral formulas that will be 
necessary for success in Differential Equations. In order to be able to deal with equations 
that involve derivatives, we need to be able to take derivatives as well as remove them. 


B.3.1 Derivative and Integral Formulas 


The following is a table of some of the basic derivative formulas covered in a Calculus 1 
course. 


Function f(z) Derivative f’(x) 
x” anyn nz 
In(x) t=! 
C constant 0 
ee e” 
et ae* 
sin(z) cos(z) 
cos(z) — sin(z) 
tan(z) sec? (a) 
arctan(x) = tan (x) a 


Similarly, we have a table for some basic integral formulas. As integration is the inverse 
operation to differentiation, this table will look like the reverse version of the previous table. 


Function f(z) Integral [ f(x) dx 
x” anyn#~-—1 eae +C 
* In(|x|) + C 
er er +C 
eat ze" Bey 
sin() —cos(x) + C 
cos(x) sin(z) + C 
a arctan(x) + C or tan“!(x) + C 


B.3.2 Derivative Rules 


The tables above only list a few simple functions for which we know how to compute the 
derivative and integral. However, there are some nice properties of derivatives and integrals 
that make this enough for our needs. 


Linearity of the Derivative and Integral 


The derivative and integral are both linear operators. This means that if we have two 
functions f(z) and g(x), and two constants a and 6, then 
df dg 


“ (af (x) + bg(x)) = aT. + bs 
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That is, we can move constants and addition and subtractions out of the differentiation, 
reducing a complicated function down to simpler functions that we know how to differentiate. 

The same is true for integration or antidifferentiation; if we have functions f(x) and g(x) 
and constants a and b, then 


[ofle)+b9l2) dv=af f(e) ae + f o(2) dt: 


Example B.3.1: Compute the following derivatives and integrals using linearity and the 
table of known formulas. 


iF ao (a tr 4 +e 3e) 
2. “ (sin(x) — 2cos(x) + 5In(x)) 
cam | 2a? tae dx 


4. [ 2cos(x) — = dx 


+1 
Solution: 


1. For this, we can use linearity and our formulas to write 


CAG ee ee eee 8 Moe oe CaS ean We 
ac rats) = 26) a (a) tg ee 


2. This one gives 


< (sin(x) — 2cos(x) + 5In(x)) = 7 (sin(x)) — 7 (2 cos(a)) + 7 (5 In(z)) 
= < (sin(x)) — re (cos(a)) + 5 (In(a)) 


5 
= cos(x) + 2sin(a#) + 


3. For this problem, we first want to simplify the expression algebraically, then integrate 
each term using linearity. 


2x3 + 4x Qn? = Axe 
TE ge Ea ae 
x x x 
4 
= fre+ de 
x 


=2fadr+af > de 
x 


= 2° +4ln(|z|)+C. 
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4. This problem uses standard linearity to get to the final answer. 


1 
[2eoste) — = i= 2 f cos(2) a3 f dx 
= 2sin(x) — 3arctan(x) + C. 


zal 


Product and Quotient Rule 


Linearity gives us a way to handle sums and differences of derivatives. What about products? 
It turns out that doesn’t work as simply, but there is still as nice formula to work it out. 
This gives us the Product Rule. If we have two functions f(x) and g(x), then 


= (fladata)) = fla) 2 + Fai) 


That is, the derivative has two terms, the first function times the derivative of the second, 
and the derivative of the first function times the second function. The product rule can also 
be used to compute the product of more than two functions; the general formula is that only 
one function is differentiated at a time and each function should be differentiated once. That 
is, for three functions, the formula is 


= (Fa)ala)h(a)) = Falapnte) + Fa) Lala) + Fala) F 


The Quotient Rule gives us a way to do the same thing, but with quotients. The formula 


here is that 
d (2) _ s@)B fog 
dx \ g(x) (gz)? 


This can also be derived using the product rule and the chain rule. It is important to get 
the order of the numerator correct, as there is a subtraction on top. For the product rule, 
the addition means that the order doesn’t matter, but if the order for the quotient rule is 
incorrect, there will be an additional minus sign in the answer. 


Example B.3.2: Compute the following derivatives. 


1. 4 (e® cos(x)) 


d vet 
3. dx (25). 
Solution: 


1. This is a direct application of the product rule. 


(e”) cos(x) 


da 
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2. This is a direct application of the quotient rule. 


d (2) x4 (sin(«)) — sin(x) # (22) 


dx xe? 


x cos(x) — 2sin(z) . 


Oats 


3. For this problem, we need to apply both the product rule and the quotient rule. Since 
the quotient rule is on the outside, we apply it first. 


d ( xe? ) _ tan(x)# (we?) — ae (tan(x)) 


dx \tan(z) (tan(x))? 
. tan(x) (23 (e”) + # (23) e”) — ae® sec*(x) 
tan?(zr) 
_ tan(x) (ze* + 3x7e”) — x¥e* sec? (x) 
tan?(x) 
ele set) are? 
~ tan(z) sin? (x) | 


ail 


Chain Rule 


The only type of function we haven’t discussed yet for differentiation is composite functions, 
and that is handled by the Chain Rule. For example, we don’t have a direct way (yet) to 
differentiate functions like sin(3z) or Seal? and the Chain Rule lets us to do that. This rule 
tells us that, for functions f(a) and g(x), we can compute the derivative of the composition 
(fo g)(x) or f(g(x)) is F 

aad (9) = FCG) 9"(@)- 

This means that we differentiate the “outside” function f, plug in the inside function, and 
then multiply this by the derivative of the “inside” function g. It requires us to identify what 
the “inner” and “outer” functions are, and then the formula gives what the derivative should 
be. This can be done in a few different ways, either moving from outside in, or moving from 
inside out. These problems are conventionally written with u(x) as the inside function, but 
any letter can be used. 


Example B.3.3: Compute the derivative of each of the following functions. 
1. fi(z) = (a? +5241)? 
2. fo(x) = cos(3x? + 1) 


3. fa(«) = (1 +sin(3z))! 
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Solution: 


1. For this problem, we take f(u) = u? and u(x) = 73+52+1, which gives that composing 
these functions gives the f, that we started with. Therefore, since f’(u) = 5u* and 
u'(x) = 3x7 +5, we have that 


fic) =f Wal (2) = 507 (62? +5) = 5 + 5a £1)" (327-45). 


2. For this case, the outside function is cos(u) and the inner function is u(x) = 3x2? 4+ 1. 
Using the same process, we get that 


f(x) = —sin(u)(6r) = —6x sin(3z” + 1). 


3. Starting from the outside, we see that we can take f(u) = u*. This makes u(x) = 
1+ sin(3zx), but we can’t differentiate this directly; it requires another iteration of 
the Chain Rule. Taking u(#) = 1+ sin(v) for v(#) = 32, we can then compute the 
derivative of each of these functions, and our original function f3(”) = f(u(v(x))). We 
can extend the Chain Rule to apply to three functions by taking it one step at a time. 
The result of this process is that 


d d 


gf (ulvl@)) = F(u(v(@)) 5 (ulo(2))) = F(u(v(a)))u"(o(a))v"(a), 


so you need to pull off one derivative at time to get to the correct computation. Thus, 
for this problem, we get that 


f(x) = 4u?(cos(v))(3) = 12u* cos(v) = 12(1 + sin(3z))? cos(3z). 


B.3.3 Integration Techniques 


Another main topic that will be needed throughout study of differential equations is various 
integration techniques. When trying to solve questions that involve derivatives, integration 
will be a very important step in that process. 


Substitution 


The substitution method for integration serves as the inverse operation to the Chain Rule for 
differentiation. Since 


the definition of the integral as an antiderivative gives that 


ii f'(u(a))ul(a) de = flu(2)) + C. 
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Integrals of this form can be computed using this formula, but it is often easier to think 
of this process in terms of “changing variables.” This means the following: If we have an 
integral that looks like 


[ tu)ul@) ae 


then we can define the variable u to represent the entire function u(x). Then the differential 
du is defined by 
du = u'(x)dx. 


Then we can substitute both u and du into the original expression to get that 
[feau'@ dx = [rw du = f(u)+C = flu(x)) +. 
The last component of this process is changing the limits of integration if a definite 
integral is being computed. The idea is that an integral in x (denoted by dz) has its limits 


also in terms of x, where as the du integral has endpoints given in terms of u. The main way 
this comes up in problems is that 


[sate ta) ae =f pee) a 


because we know that wu is written in terms of x as u = u(x). Thus if we plug the x endpoints 
into this function, we will be the new wu endpoints. 


Example B.3.4: Compute the following integrals using substitution. 
ip [ costa dx 


2: [ esin(a0? +1) dx 


2 2 
3 
fie 
9 ow +4 


Solution: 


1. For this situation we want to set u = 4x, because then the integrand, once we make 
the change of variables, will be cos(w), which we know how to integrate. With this, we 
have du = 4 dx, which we can rewrite as dx = s du. Plugging all of this in gives that 


1 1 1 1 
[ cost die [costw)y i= ‘I [costu) eS ii sin(u) +C = A sin(4r) + C. 


2. For the same reason, we want to set u = 32?+1 to make the resulting integral sin(u) du. 
In this case, we have du = 6x dx or x dx = 7 du. Plugging all of this in, we get 


1 1 1 1 
[ esin(3e%H1) age [snes dic= =f sintw du = a cos(u)+C = ars cos(3z?+1)+C. 
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3. We can follow the same logic here as for the previous examples, but since we have a 
definite integral, we also need to switch the limits of integration. In this case, we want 
to pick u = x? + 4, which gives du = 32? dx. This gives the resulting integral as 


3x? 1 
fame- [ze 


For the limits of integration, we take the function u(x) = x? +4 and plug in the original 
values of 0 and 2. This gives the value 4 and x = 0 and the value 12 at x = 2. Therefore, 
the result of this computation is 


2. a2 12 4 r 
/ SLA d= / ; du = In(|u]) |4°= In(12) — In(4) = In(8). 


_| 


There can be some cases where these techniques will not work, because the u’ term that 
you are looking for doesn’t quite appear in the expression you are trying to integrate. In 
cases like this, you may need to use some more complicated methods (like trigonometric 
substitution) or connect to inverse trigonometric integrals or other known formulas. 


Integration by Parts 


Integration by parts is the method used to handle integrals of a product of functions. Like 
the substitution method is the inverse of the Chain Rule, integration by parts is the inverse 
of the product rule. There are two main formulas that are used for this process. For two 
differentiable functions f(x) and g(x), we have 


/ f(a)d (a) de = f(a)9(x) — : soap ees 


fudo=w— fv du 


which matches the original form after setting u = f(x) and v = g(z). 
The most important part of this process is picking the appropriate functions for u and v 
in this formula. The general rule is given by the following list 


The other form is 


e Logarithmic functions 

e Inverse Functions 

e Algebraic or Polynomial Functions 

e Trigonometric Functions (sine and cosine) 


e Exponential Functions 
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and you want to make wu, the function that you are differentiating, the one that is higher on 
the list. The main reason for this list is that integration is much harder than differentiation, 
and so we generally want to integrate the part of the product that we have a formula for. 
This is why logarithms and inverses are on the top; we know how to differentiate them, 
but integration is difficult or impossible. Polynomials are good for both differentiation and 
integrals, but the benefit of differentiating them is that they eventually disappear, leaving 
us with an integral that we know how to solve. For example, x? becomes 2x, and then 
differentiating a second time gives 2, which is just a constant and can be removed from the 
integral. Trigonometric and Exponential functions are interchangeable, they are easy to 
differentiate and integrate, and they don’t go away if we keep applying either operation. 

This method can also be performed mutliple times by redefining wu and v and applying 
the same process to the integral that remains on the right-hand side. When doing this, it 
is important not to reverse the roles of u and v, because then the process will just undo 
what was done in the first step. There are also some cases where circular reasoning is used, 
integrating by parts twice to get to the same expression on both sides of the equal sign, which 
can then be solved for. One of those will be shown in the examples below. 


Example B.3.5: Compute the following integrals. 
1. f xsin(2x) dx 
Ds (agre de 
By ee" cos (37) ae 

Solution: 


1. Based on our list, we should choose u = 2, as it is a polynomial function. This means 
that dv = sin(2z) dx. From this, we get that du = dx and we compute v by integrating 
sin(2x) dx, which requires a substitution. This results in v = —4cos(2zx). Thus, the 
integration by parts formula gives 


i note = (-3 cos(2) 2 / (-3 cos(2) ae. 


This last integral we can compute directly, again requiring a substitution. Thus, the 
final answer is 


1 
J esin(22) eS = cos(2a) + i sin(2x) + C. 


2. By the same argument as the first example, we want to pick u = 3x? so then du = e** dz. 


We can then compute that du = 6x dx and v = ze". Thus, integration by parts gives 


1 1 3 3 
24a = 2, —,4@ ) _ = 4a — 240 _ Sp pst 
[x en gr ae (Fe ) i (Fe ) (6x dx) gre [5 dx. 


This last integral is not something that we know how to compute. However, it looks 
like a product, so we should be able to work it out using integration by parts. We can 


A84 
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set u = 3x and dv = e*” dx. This is the same dv as before, which is good. If we had 


picked dv = Sr dx, we would have just gotten back to where we started. From these 
choices, we get that du = 3 dx and v = ze*, Integration by parts then gives that 


3 3 3 
[pee dx = gre - i: ac dx. 


Now we can compute this last integral, which will give another factor of ‘, resulting in 


3 3 3 
i 5ree dx = gre — ll +C. 


Finally, we can combine this with our first integration by parts step to get that 


2 Ax 2 4a Aa Ax 
= = tC. 
fo e dx rag e are 39° 


. For this example, we have both an exponential and a trigonometric function. We can 


pick either one to be u and dv, and as long as we are consistent with that choice, we 
will get to the correct answer. For this, we will choose u = e?” and du = cos(3z) dx. 
From these, we can compute that du = 2e?" dx and v = 5 sin(3z). Thus, integration by 
parts tells us that 


1 2 
fe cos(3x) dx = aoe sin(3a2) — ii x sin(3x) dx. 


This new integral is again a product, so we need to handle it using integration by parts. 


To do this, we are going to pick u = ze and dv = sin(3x) dx. Note: If you pick 


u = sin(3z) and du = zee dx, the second integration by parts will just give that 


ee cos(3x) dx = fe cos(3x) dx 


which does not help in solving the problem. With the correct choice of u and dv, 


u = 2e”* and dv = sin(3z) dx, we have that du = $e?” dx and v = —} cos(3z), so that 


3 
integration by parts tells us that 
2 2 4 
[5e sin(3x) dx = —<e?* cos(3x) — - —~e”* cos(3x) da. 
3 9 9 
Combining this with our first integration by parts gives 
2x L 22:58 2 22 os 
e cos(3x) dx = ae sin(3a2) — ae sin(3x) dx 


1 2 4 
= 3° sin(3a) + ac cos(3x) — / ae" cos(3x) dx 


1 2 4 
is cos(3x) dx = 3° sin(3a) + ac cos(3x) — 9 [2 cos(3x) dx. 
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In this case, we can see that the integral on the left matches the integral on the right. 
If we combine these on the left side, we get 


13 1 2 
5: e*" cos(32) dr = Bee sin(3a) + sc cos(32) 


which then allows us to solve for the answers as 


3 2 
2x 2% 22 22 
fe cos(3x) dx = 13° sin(32) + ig” cos(3z). 


Partial Fractions 


Another integration technique that shows up frequently when dealing with rational functions 
is the method of partial fractions. This method works around decomposing a rational function 
into forms that we are able to integrate. For example, we do not have a formula or method 


to compute the integral 
3 
————. dz. 
/ v2—-—x-2 v 


since there is no simple function whose derivative is a, What functions like this can we 
integrate? 


Example B.3.6: Compute the following antiderivatives 


@) [a o) [age © faze 


Solution: 


(a) This integral can be computed by a substitution u = x — 2, 


1 1 
/ da = f © du=Injx 2) + 
L— 2 u 


(b) This integral is another substitution, but the goal here is arctangent, not a logarithm. 
We let u = 2/2, so that du = 1/2 dx, and then 


1 1 1 1 1 x 
aCe ee Pate Oe 
=e lea U sfoae 5 arctan 5 +C 


(c) With an x on top of the expression, we can now use a substitution u = x? + 9 to solve 


the integral. 
x 1 1 1 
Jeu 5 [5a 5 in |a" + 9] + 
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_| 


So, we can handle these types of integrals, but that doesn’t necessarily help us with the 
initial one. Let’s take a look at another example. 


‘| | 
— — dz. 
fa r+i1 ‘i 


Solution: This integral can be computed by splitting it into the two terms present. Each of 
those we know how to evaluate using the previous example. Thus, we have that 


Example B.3.7: Compute 


1 1 jz —2| 
= =| — 2|) -—1 1 =| : 
la sei” n(|x — 2|) —In(Ja+1]/)+C n(FoZ) +c 


a 


This is great! However, we can compute that, by adding fractions 


1 1  («@+1)—(«-2) 3 


e—-2 @+1  (¢-2)(e@+1) 2-2-2 


So this gives us a way to compute the original integral of this section, and we now know that 


3 |x — 2| 
= —— d=! : 
(fae ° (ea) te 


This gives an idea for how we may be able to evaluate integrals of rational functions. In 
the case of the integral above, we would need to figure out a way to convert between 
3 1 1 


and = : 
z?—xz—2 z—-2 -2+1 


that is, we need to split the complicated fraction into the smaller, simpler partial fractions 
that we can integrate. Based on our work in Example B.3.6, we know that we can integrate 
functions that have a linear term in the denominator and a quadratic term in the denominator, 
and the process of putting these fractions together into a single term involves multiplying 
the individual denominators together. This gives the motivation for the method of partial 
fractions for integrating rational functions: 


1. Factor the denominator of the function we need to integrate. Any polynomial can be 
factored into linear terms (terms like x — a) or irreducible quadratic terms (terms like 
x? +4 or x74 27 +5). 


2. Write an expression with unknown coefficients for each factor in the expression. If it 
is a linear term, it will need just a single constant, but if there is a quadratic term, it 
needs a numerator of the form Ax + B. 


3. Solve for the necessary constants (more on this later). 


4. Integrate each of the resulting expressions, which are all forms where we know the 
antiderivative. 
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5. Combine the terms into a single expression. 


The process is best shown through an example. 


/ 3x +1 q 
——_—__—__——__ dz. 
ve? — £2 — 127 


Solution: We start by factoring the denominator. We can factor an x out of each term, and 
then the resulting quadratic can be factored. Since 


Example B.3.8: Compute 


gp — 2? —12¢ =2(2+3)(a —4) 
we want to figure out coefficients A, B, and C' so that 


bz +1 A BB, ¢e 


oe —o2—1l2e cg ec+3 2-4 


where we have one term per factor of the denominator. In order to find these constants, our 
first trick is to multiply both sides of this equation by the entire denominator on the left. 
This gives 
32 +1= A(x+3)(x2 — 4) + B(x) (a4 — 4) + C(x) (ze + 8) (B.1) 

where we have cancelled the appropriate terms from the top and bottom of each expression. 
One way to go from here to the constants is to expand out the right-hand side and recognize 
that for these two sides to be equal for all x, the coefficient of x”, x, and the constant term 
must match. This will result in solve a system of 3 equations. 

An easier approach to doing this is to plug values for x into each side, and to pick those 
values cleverly. One clever choice for (B.1) is to set x = 0. If we do that, both the B and C 
terms will go away, because they are multiplied by zero. Thus, if we plug in zero, we get 


1=+12A+0+0 


which implies that A = —1/12. For the next term, we can plug in —3 to make the x + 3 terms 
go away, resulting in 
-8 = B(-3)(-7) 


so that B = —8/21. Plugging in x = 4 gives 
13 = C(4)(7) 
so that C' = 13/28. Therefore, we can write that 


32 +1 =e, =a, 8 
g—ag?-12r e3 - od 


Then, we can integrate both sides to get that 


3 1 1 8 13 
/—: de =~ In(le) — 5 In(|e + 3)) + 5 In(|e — 4) +. 


—In 
zg? — 7? — 122 12 21 
_| 
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The same type of approach applies if there are irreducible quadratics in the expression. 


/ 2x? — 6 q 
o—a24+4¢—4 
Solution: The denominator can be factored as (x? + 4)(x — 1), which can be determined 
by grouping. This means that to do the partial fraction decomposition, we need to find 
coefficients A, B, and C’ so that 
227 —6 _Ar+B, C 
g—a2+4¢—-4 2244 


Example B.3.9: Compute 


e=1- 


Note that the x?+4 term has Ax+B on top instead of just A. This is because the term on the 
bottom is a quadratic, and there will always be a number of coefficients on top that matches 
the degree of the term in the denominator. By multiplying both sides by the denominator 
will give the equation 


Qn" — 6 = (Ax + B)(x — 1) + C(a* +4) 


where we need to find the appropriate constants. In this case, we can plug in x = 1 to 
determine that —4 = 5C' or C = —4/s. However, there is no value we can plug in to make 
x? +4=0. We could use complex numbers here, but assuming we don’t want to do that, we 
can plug in any two numbers and go from there. Plugging in x = 0 is nice because it makes 
the A term go away, resulting in 
16 

—6 = B(-1)+ C(4) =-B- = 
which we can solve to get B = 14/5. Finally, we can plug in any other number for x to get an 
equation to solve for A. Let’s use —1 to give that 


Spe (-4+ 4) (—2) + (-§] ()=24- 2-4 


which gives that A = —'4/s. Therefore, we can write 
2 
2x* — 6 = —14/s5x + 14/5 | —4/s . 
ge —a2?+4r—4 x2+4 x—1 


Therefore, we can write the integral we want to compute as 
i 2x? — 6 ne / —Misa + M/s | —4/s5 ae 
x —2?+4r—4 g2+A4 x-1 
14 x 14 1 4 1 
=— dz 4 dx — = | ——d 
sfem@+sfene@ f/m 


7 7 4 
= —z In([e” +4]) + 5 arctan (=) = In(|z — 1]) +C 


_| 


There are a few extra complications that can result from using this method. 
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. If there is an irreducible quadratic like x? + 2x + 5 in the denominator, we will want 


to separate that out and complete the square before integrating. In this case, we 
have 2? + 2x +5 = (2+ 1)?+4, so we will want to use A(z +1) + B when solving 
for coefficients (to make the u-substitution work better), and will get a slightly more 
complicated result. 


. If there are repeated factors, like (2 — 1)? in the denominator, we need to include one 


term in the partial fraction expansion for every power of that factor. For instance, the 
expansion should look like 


1 A B C D 


GAGs. el 28 Goa G3. 


. If the rational function has an equal or higher degree in the numerator than in the 


denominator, we will need to do long division to remove a standard polynomial (which 
we know how to integrate) and a proper rational function that can be integrated using 
partial fractions. 


Combining all of these techniques together will allow us to integrate pretty much any 


rational function that we need for a given application. 
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Answers to Selected Exercises 


0.1.5: Compute 2’ = —2e~** and x” = 4e~*. Then (4e~*) + 4(—2e7*") + 4(e~*") = 0. 
0.1.8: Yes. 

0.1.10: y=2" isa solution for r = 0 and r = 2. 

0.1.13: Ci = 100, Cy = —90 

0.1.15: y= —9e° 

0.1.17: a)x=9e* b) x =cos(2t)+sin(2t) c) p=4e*% d) T = 3sinh(2z) 

0.2.2: a) PDE, equation, second order, linear, nonhomogeneous, constant coefficient. 

b) ODE, equation, first order, linear, nonhomogeneous, not constant coefficient, not au- 
tonomous. 

c) ODE, equation, seventh order, linear, homogeneous, constant coefficient, autonomous. 
d) ODE, equation, second order, linear, nonhomogeneous, constant coefficient, autonomous. 
e) ODE, system, second order, nonlinear. 

f) PDE, equation, second order, nonlinear. 

0.2.6: equation: a(x)y = b(x), solution: y = a. 

0.2.7: A= ork=1 

0.2.9: b) First order with three components. 

c) Third order with one component. 

d) The product is three in both cases. (1 x 3 =3 x 1). 


1.1.4: y=e"+249 
1.1.11: 170 
1.1.15: The equation is r’ = —C for some constant C. The snowball will be completely 


melted in 25 minutes from time t = 0. 
1.1.16: y=Az?+ Br? +Cx+D, so 4 constants. 


© Fe 4 


solution such that y(0) = 0. 
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1.2.9: a) y’=cosy, b)y’=ycos(x), c)y’=sinz. Justification left to reader. 
1.3.2: «= (3t—2)'% 
1.3.4: ¢=sin"'(t+1) 
1.3.8: a) Y=22+C b) y=2V27+3 chy=-2V2z?+1 
1.3.9: Ifn #1, then y = ((1 —n)xt+ es, If n = 1, then y = e*. 
1.3.12: y=Ce™ 
1.3.15: 2=e"+1 
L318), a Pus te! 
1.3.21: sin(y) = —cos(x) +C 
ib tees ool 
1.3.24: Y= Tone 
1.3.33: The range is approximately 7.45 to 12.15 minutes. 


1.3.34: a) x= ae b) 102 rabbits after one month, 861 after 5 months, 999 after 10 


months, 1000 after 15 months. 
1.4.13: y=Ce-™ + 1/3 
1.4.19: y = 2es*)+1 4 4 


1.5.16: Yes a solution exists. y’ = f(x,y) where f(z,y) = xy. The function f(z, y) is 
continuous and ol = x, which is also continuous near (0,0). So a solution exists and is unique. 


(In fact y = 0 is the solution). 
1.5.17: No, the equation is not defined at (x, y) = (1,0). 


1.5.18: Picard does not apply as f is not continuous at y = 0. The equation does not have 
a continuously differentiable solution. Suppose it did. Notice that y’(0) = 1. By the first 
derivative test, y(x) > 0 for small positive x. But then for those x we would have y’(x) = 0, 
so clearly the derivative cannot be continuous. 


1.5.19: The solution is y(x) = ee f(s) ds + yo, and this does indeed exist for every x. 
1.6.7: Approximately: 1.0000, 1.2397, 1.3829 

1.6.9: a)0,8,12  b) #(4) = 16, so errors are: 16, 8,4. c) Factors are 0.5, 0.5, 0.5. 
1.6.10: a)0,0,0 b) «=0 isa solution so errors are: 0, 0, 0. 


1.6.12: a) Improved Euler: y(1) = 3.3897 for h = Ya, y(1) = 3.4237 for h = 1/8, _ b) 
Standard Euler: y(1) ¥ 2.8828 for h = Ya, y(1) © 3.1316 for h = 1/8, c) y= 2e7-—a2-1, 
so y(2) is approximately 3.4366. d) Approximate errors for improved Euler: 0.046852 for 
h = 1/4, and 0.012881 for h = 1/s. For standard Euler: 0.55375 for h = 1/4, and 0.30499 for 
h=1'/s. Factor is approximately 0.27 for improved Euler, and 0.55 for standard Euler. 


1.7.4: a) 0, 1, 2 are critical points. b) x = 0 is unstable (semistable), x = 1 is asymptoti- 
cally stable, and x = 2 is unstable. c) 1 


1.7.9: a) There are no critical points. _ b) co 


1.7.11: a) a isa stable critical point, 6 isan unstable one. b) a, c)a,  d) oo or DNE. 


2 
1.8.3: a)“ =kse(M—z2)+A_ b) kM+ ces +4Ak 
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1.9.3: a) e™%+sin(z)=C  b) a? +2y—-2y2=C che*+e%=C dd) a? 4+3ry+y=C 
1.9.10: a) Integrating factor is y, equation becomes dx + 3y?dy=0.  b) Integrating factor 
is e*, equation becomes e* dx —e~¥dy =0. __c) Integrating factor is y?, equation becomes 
(cos(z) +y)dx+ady=0.  d) Integrating factor is x, equation becomes (2ry + y?) dx + 
(x? + 2xy) dy = 0. 


1.9.15: a) The equation is — f(x) dx + 4 dy, and this is exact because M = —f (x), N = 
gy) 


AChE soM,=0 = Ne. ‘b) =e da+< dy = 0), leads to potential function F(z, y) = —=4Inlyl, 
solving F(x, y) = C leads to the same solution as the example. 


1.10.5: 250 grams 
1.10.9: P(5) = 1000e2%5—0-95x5" — 1000e875 = 6.31 x 10° 


1.10.10: Ah’ = I —kh, where k is a constant with units m?s. 


1.11.4: @ = .123. The alpha value used before noise was added to the data is 0.124, so very 
close, but not identically the same. 


1.11.5: a) a= 4.03 x 10-°, soa 0. c) K = 324.07 and a = 5.061. 


1.12.2; y= 5 
3-2? 


1.12.4: y= 


22 


1.12.9: Y= (7e# + 3x + 1) 1/3 


1.12.13: y = \/x? — In(C — 2) 
2.1.5: Yes. To justify try to find a constant A such that sin(x) = Ae* for all x. 


2.1.6: No. e®+? = e’e*. 
2h Yo 5 


2.1.13: y=Ci In(x) 4205 
Qo Ols ap Cye(-2+V2)# 4. Cye(-2-v2)« 


2.1.22: y= se 1 Sate) ex 


2.1.23: y= Fem + Baa ee 

2.1.24: y” — 3y'+2y=0 

2.1.25: y”—y'—6y=0 

2.2.5: 3,/2cos (2a — 7) 

2.2.13: y = e~*/4cos((v7/a)x) — V7e~*/4 sin((V7/4)z) 

2.2.14: z(t) = 2e~* cos(t) 

2.2.17: There is no such equation. The two roots will always be complex conjugates, which 


means the exponential parts will match, and the trigonometric functions will have the same 
argument. 


2.3.4: y = Cie?” + Chre* 
2.3.10: c) y(z) =Cir+ O25 
2.3.11: c) y(z) =C12+Q4 
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2.3.12: c) y(x) = Ci2? + Cox° 

2.4.5: k = 8/9 (and larger) 

2.4.8: a) k=500000 _ b) a ~0.141 c) 45000kg~ d) 11250kg 

2.4.10: mp = ; If m < mp, then the system is overdamped and will not oscillate. 

2.4.11: a) 0.05/”+0.11'+(1/s)I=0  b) I= Ce~*cos(V3t — y) or I = Cye~* cos(V3t) + 


Coe sin(/3t) cc) I= 10e* eee ame —t sin(V/3 t) 
. __ —16sin(3x)+6 cos(3z) 
2.5.5: y= 73 
2.5.9: y(r) =2?—4r+6+e (x —5) 
2.5.12: a) y= 2et3u0 Se b) y = C, cos(V2z) + Czsin(V2x) + de +5e0—9e 
. __ 2ae*—(e* +e~*) log(e?* +1) 
2.5.23: y= ie 
2.5.25: y= —Mmet) 1 Cyev?# 4 Cye-V? 

: = (wo—w*) Fo QwpFo : tas 
2.6.6: Lsp Sane dala 7, cos(wt) + Goat tm(utaa)? sin(wt) + 4, where p = = and 
Wo = . 

2.6.9: w= Y2~ 0.984 Cw) = 1% ~ 2.016 

2.6.12: a)w=2  b) 25 

2.7.3: y= Cie” Cox? Cae" Chr Cs 

2.7.8: a) r?—3r2+4r—12=0_ b) y”—3y"+4y’-12y=0 c) y= Cye®* + Co sin(2x) + 
C3 cos(22) 

2.7.10:  y(x) = Cye* + Cye~* + Cze~* cos(2x) + Cye~ sin(22) 


2.7.19: No. ee — e**1 
PE Pe 
be multiples of each other.) 


= 0. 


Yes. (Hint: First note that sin(a) is bounded. Then note that x and z sin(a) cannot 


2.7.24: y=0 
2.7.28: y/” —y" +y'-—y=0 
3.1.5: a) /10 b) V14_ c)3 
9 23 5 —4 3 —8 
3.1.7: a) el ») |S) éj EA ale ‘ H »|5) 
cs 
=1 V6 
3.1.9: a) Fd byilce) (4. +) 
7) is 
Ve 
3.1.14: a) 20 b) 10 c) 20 
3.1.18: a) (3,-1) b) (4,0) ¢) (-1,-1) 
5 -3 0 
3.2.2: 3B) i F : Dy. 3. 0> <6 
= eee 
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3.2.4: a) ie al b) E . 


bs a 18 18 12 1: To 36° 14 Ho 19 
3.2.6: a) a a BYR G. Beh) Sepa oa ee) way ee a0 
34 48 —2 13 38 20 28 1 9 


3.2.11: a) [1/2] b) fi 1 c) B 2 d) a A 


—-1 0 0 0 
/4 0 O 
V/o 0 0 4 0 O 
3.2.13: of v1 br) }0 ys 0) |p 4 ve 
ys OO: 20 30 
1 0 0 Le 06 50> 2715 
3.3.2: a) F : 1 b) i | 0) i A a) {o 1-1/3] ©) lo 1 0 —2/15 
0 0 0 00 1 —8/5 
i 0 1/2 0 
0 0 0 0 12 3 0 
f) 0 1 172 1/2 g) | h) | 
0 0 0 0 00 0 0 000 1 


3.3.4: a) 7) = —2,22=7/3 b)nosolution c)a=-—3,b=10,c=-8 _ 4d) 23 is free, 
t= —14 373, x2 =2-— 273 

doe, y= 4, te] 3, 63 = 2a = 3 

3.3.6: Ly = —4A, t= —l, w3 = 1, LA = 2 

3.3.7: No solution exists. 

3.3.8: Infinitely many solutions of the form x, = 19t — 37, ro = 37 — 20t, x3 = 15t — 37, 
x4 = t for any real number t. 

3.3.9: There is no solution. 

3.3.10: There are infinitely many solutions of the form x; = 2 —t, x2 = 4 — 2t, x3 = t for 
any real number t. 

3.3.12: The work is not correct. It looks like the author used row 1 to try to cancel the 
second column from rows 2 and 3, which we can not do. The correct method would be to use 
row 2 to cancel row 3, resulting in a solution x; = 9, v2 = —25, and x3 = 10. 


34.2: 8) 3° “by Lb ~e)-2 


S4coe a) [30-0]. (0 ae Oy sir Ora) bye iy aie, ene) PRO) Gs al 73) 
7] [-1] [7 1 0] [3 

BA.62 od) 715) [Ts 18 b) }1 c) 16], |3 
fal ee Wea i 2 4) |7 

3.4.7: 3 

3.4.8: 4 
3 0 

3.4.10: | 1],] 3 
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l l 1 1 3) 3) —1 
3.4.12: a) H : H dimension 2, b) {1}, |1] dimension 2, c) |/3],}—-1],] 3 
1 2 1 3) —4 
2 2 1 1 0 
dimension 3, d) |2} ,}2] dimension 2, e) H dimension 1, f) {0} , |1] dimension 2 
4 3 0 2 


3.4.14: 


a) Put the vectors as the columns of a matrix and row reduce. If there are any non-pivot 
columns, the vectors are linearly dependent. 


b) No, there can be at most three pivot columns, so with four columns, one must be 
non-pivot. 


c) Yes, there is no reason you can’t have all of the two columns being pivot columns. 


d) Put the vectors as the columns of a matrix, and look for solutions to AZ = b. We need 
the rank of this matrix to be at least 3. 


e) Yes, the matrix with four columns can have rank three. 
f) No, it is impossible for a matrix with only two columns to have rank three. 
3.4.15: 
a) The rank is 2. 
b) No, it is not in the span. 
c) Yes, it is in the span, because the first vector is exactly b. 
d) This says that these two spans are not the same. We can not use the row-reduced 


matrix in order to figure out if something is in the span. We need to use the pivot 
columns to go back to the original vectors to simplify the span. 


e) 


f) No, it is not. If we add the two rows together, we get [1 0 — 1/2] and we have no way to 
cancel out that last term. This suggests that we can use either the rows of the original 
matrix or the rows of the row-reduced form in order to work out the span of the rows. 
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3.5.3: a)—-2 b)8 c)O d)-6 e)-3 f)28 g)16_ h) —24 
3.5.5: a)3 b)9 c)3 = d) V4 


3.5.6: -10 
3.5.7: 6 
3.5.8: 6 


3.5.10: Rank is 3. Therefore A is not invertible (since the rank is not 4), and there are 
non-zero solutions to Ax = 0. 


1 
3.5.11: Rank is 3. Therefore A is invertible, and there is exactly one solution to AY = }1}], 
1 
1 
namely A~! | 1}. 
1 


3.5.12: -1. The only solution is # = 0. 

3.5.13: 2. The colums are linearly independent. 

3.5.14: 8. There is exactly one solution, found by row reduction or multiplying by A7?. 
3.5.16: 1/12 

3.5.19: land3 


3.6.1: Ay =—-2, 0, = PD) a4 w= Ey 


1 =) 
a 1 ze 0 

3.6.2: w= -2, = [j], a= —4 a=|i}. 

ee Be yi ee || Aes ae 

Stee 1 —4, U1 = 9 |? 2 Oy US aes 

3.6.4: a a=342i a= | 74) 3-24 a= [PT 


3.6.5: ma-t+6h=[_ Jde=-1-a = |_y | 


1-4 ee 
S66. Os 94 Oke a | ye ae 
4 4 
8.6.7) \y=4, c= ) 
3 1 
3.6.8: M1 = —3, Uu= | 
2 
0 0 a} 
3.6.9: M1 = 2. U1 = {1 ; 2 = 1, Uy = |-1 5 A3 = A, U3 = |-3 
0 1 =2 
1 0 3 
3.6.10: M1 = —A4, v1 = 3 ; r2 = —3, 3° = —1 5 AZ = —-1, U3 = 0 
—3 1 1 
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0 0 1 
3.6.11: Ay = 14 31, v= 2 , Ag = 1— 32, U2 = 2 ,A3 = —2, eS 2 
—l+i —l1-1 —2 
1 0 
3.6.12: A, =2,0,= |] 1 |, Ae =1, v2 = |—2] (double root) 
—1 1 


3.6.13: A, =—2, %) = lak eae Ee | 


3.6.14: A, =2,%,= 
3.6.15: 


a) (—2—2) [(5 — A)(1 — A) + 12] +36 = (2—A)(5—A)(1—A) —12(24 A) —12(—3) which can 
be regrouped as (2—2)(5—A)(1—A) —12(2+A) —12(—3) = (2—A)(5—A)(1—A) +12(1-A) 
and can then be factored as (1 — \)(A* — 5A + 2A — 10 + 12) = (1— A)(A— 1)(\ — 2). 


b) r; = 1 with algebraic multiplicity 2, and ry = 2 with algebraic multiplicity 1. 

c) [3 4 — 4]. Geometric multiplicity is 1. 

d) [1 0 — 1]. Geometric multiplicity is 1. 

e) [—1/3 10]. There are many answers here, and they will satisfy v2 = 1 and vi+v3 = —1/3. 


3 0 O 
3.6.19 0 —2 1 
OF 208 - 2 

is ; = 1 -1] fo 

3.7.3: a) : b) |-1 ren ae dD): | Os hg Pt 
0 3 

0 4 0 sail 0 eal 


—-l 0O = 0 


3.7.12: (i) Trace is 1, determinant is -2. Eigenvalues are -1 and 2. 
(ii) Trace is -2, determinant is 10. Eigenvalues are —1 + 33%. 

(iii) Trace is -2, determinant is -8. Eigenvalues are -4 and 2. 

(iv) Trace is -8, determinant is 16. Eigenvalue is —4 repeated. 
3.7.13: (i) Trace is 6, determinant is 6. Eigenvalues are 1, 2, and 3. 
(ii) Trace is -9, determinant is -39. Eigenvalues are -3 and —3 + 21. 
(iii) Trace is 1, determinant is -24. Eigenvalues are 2, 3, -4. 
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4.1.6: y= Cre*”, yo = y(x) = Coe” + De®”, ys = y(x) = Cze” + De” 

AAGs = he = ey ee * 

ANAQ? <2 0 05 HS 0s, 2S at 

41.11: ytymty=tytm-y=O, = Ys, = Y 

4.1.17: x, = 22 = at. Explanation of the intuition is left to reader. 

4.1.19: a) Left to reader. b) x, = F(t2—-%1), T) = Ft1—Fet2. c) As t goes to 
infinity, both x; and x2 go to zero, explanation is left to reader. 

4.1.20: a) (i), b) (iii), c) (ii) ~— Justification left to reader. 


4.1.21: a) (iii), b) (ii), ce) (i) ~—s Justification left to reader. 


4.2.7: —15 

4.2.11: —2 

4.2.13: Ly = 3, v2 = A, £3. > —3. 

4.2.14: Infinitely many solutions of the form 7, = i + at, Ly = it — 2, x3 =t for any 


real number t. 
4.2.15: No solution. 
4.2.16: 2, = —2, %2=1, x3 = —4. 


4.2.19: #=[15] 
a OO 
4.2.22: a) | ),]  b) 01 | 
~~ fee 
4.2.24: M = 1, Uy cE d2 =2 


4.2.25: ri =) =e a3 21, Oh = ee ‘| r2 = —2 a 21, Vo = ie i ' i 


1 1 
4.2.26: A, =4, 0, = 1], A. = —2, % = 13], As = —3, vs = |-1]. 
0 0 


4.3.6: Yes. 


iw 

ow 

—_ 

j=) 

i) 
4 

iS 
0g 

lo) 

e 

—_ 

f- 

i) 

a 
on 
=) 

| 

KH 

iS 
0g 

lo) 

F 

(o) 

Q 

ot 

o) 

5B 

iS 

aT | 
RO 
ud 
| ena | 
ORO 
m1 
Lo 

ud 


b) €=C, [ofe# +e, [1] +05 | B]es 
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0 —1 3 
4.4.14: Z(t) =C, }0| e*+C.} 1 | e%+C 3 |-2| e* 
1 —3 —2 
2 y) 1 
4.4.15: £(t)=C, |0| eo %+C, 13] e+ Cs |2| e* 
1 1 0 


45.4: 2 = on a0) As c, bel 


cos(t) 


4.5.6: a) Eigenvalues: a a Eigenvectors: Fear Fan 


b) @= Ce” ee | +Guet” | pee, 


cos (Yt) +3 sin (2) sin (3 te V3 cos (¥3 | 
4.6.11: a) 3,0,0 b) Nodefects. c)#=C, E e* + Cy | 3] +C%3 H 


4.6.13: a) 1,1,2 
b) Eigenvalue 1 has a defect of 1 


t=Ci[tle+er([of+e[ i |Jet+os| de 
4.6.15: a) 2,2,2 
b) Eigenvalue 2 has a defect of 2 


0) = 0, [fer +er([a] +e|i])errea([3] rela] +3 [fe 
4.6.19: A=([33] 
4.6.20: (a) Nodal Source, (b) Saddle, (c) Spiral Sink, (d) Center, (e) Spiral Source, (f) 
Saddle. 
4.7.5: a) Two eigenvalues: +V/2 so the behavior is a saddle. _b) Two eigenvalues: 1 and 2, 
so the behavior is a source. _c) Two eigenvalues: +27, so the behavior is a center (ellipses). 
d) Two eigenvalues: —1 and —2, so the behavior is a sink. e) Two eigenvalues: 5 and —3, 
so the behavior is a saddle. 
4.7.7: Spiral source. 
4.7.8: a) Nodal source c) Spiral source c) Saddle cc) Nodal sink e) Spiral sink _f) 
Improper nodal sink 
4.7.13: 


The solution does not move anywhere if y = 0. When y is positive, the solution moves (with 
constant speed) in the positive x direction. When y is negative, the solution moves (with 
constant speed) in the negative x direction. It is not one of the behaviors we have seen. 
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Note that the matrix has a double eigenvalue 0 and the general solution is x = Cit + Cy and 
y = C,, which agrees with the description above. 

4.7.14: (i) T =—6, D=8. Nodal sink. All points nearby are nodal sinks. 

(ii) T = 2, D=—3. Saddle. All points nearby are saddles. 

(iii) TZ = 0, D=1. Center. Points nearby are all spirals, but they could be asymptotically 
stable, centers, or unstable. Stability is unknown. 

(iv) T = 6, D = 10. Spiral source. All points nearby are spiral sources. 

(v) T = —8, D = 16. Improper nodal sink. All points nearby will be asymptotically stable, 
but they could be nodal sinks, improper nodal sinks, or spiral sinks. 

(vi) T = 4, D=3. Nodal source. All points nearby are nodal sources. 

(vii) T = —4, D = 13. Spiral sink. All points nearby are spiral sinks. 

(viii) T = 2, D = 1. Improper nodal source. All points nearby will be unstable, but they 
may be spirals, nodal sources, or improper nodal sources. 


4.8.6: The general solution is (particular solutions should agree with one of these): 
x(t) = Cye* + 4C ge" — 4/3 — 5/54, y(t) = Che** — Coe* + #/6 + 7/216 

4.8.8: The general solution is (particular solutions should agree with one of these): 
x(t) =Cye'+Coe*+te’, y(t) = Ce’ — Coe + te’ 


4.8.10: x*=[t{] (2c! -—t- 1) ae ale 


AD AS: f= || (a; cos(V3t) + by sin(V3t)) + 3, (a2 cos(/2t) + by sin(/2t))+ 


4.9.9: x2 = (2/5) cos(,/1/6t) — (2/5) cos(t) 


(( et em ; ( oa ) ev t sont | 
140 ' 120/6 ' \ 140 ' 120/76 60 70 
cos(t) 


+ a cos(2t) + a eae as ) 


ate 
e "-e est4e—t 
2 2 


Qe! —2€?¢ et 2e7* et 
; t 9.3t : 
2e3t —5e2t +4 3e! dene se e3 +5e2' —3et 
2 2 


) 
tien e—t_e3t 
4.10.4: e4= ae | 


2e3t —4e?4 + 3e! aa Be oe” 


4.10.6: a) e4= ee ae b) #= baal 


tet — (1—-t) e?# (2—t) e?# 


. 142t+5t? 3t+6t? 0.1A ~ [1.25 0.36 
4.10.15: 94 At? ea en" & [034 138] 


a Ee — 242 4(3") -— i ; — 2(3") 2(3") -2 
5(2”) — 5(3")  5(2") — 4(3") a=3rtt gut 2 


c) E | if n is even, and 1 if n is odd. 


504 ANSWERS TO SELECTED EXERCISES 


5.1.3: (i) isc), (ii) isa), (iii) is b) 

5.1.5: a) Critical points (0,0) and (0,1). At (0,0) using u = x, v = y the linearization 
is u’ = —2u — (Yr)v, v' = —v. At (0,1) using u = 2, v = y — 1 the linearization is 
ul = —2u+ (Yr)v, uo! =v. 

b) Critical point (0,0). Using u = x, v = y the linearization is u’ = u+v, v' = u. 

c) Critical point (1/2,-1/4). Using u = x — ¥/2, v = y+ V4 the linearization is u’ = —u +, 
v =ut. 

5.1.11: Critical points are (0,0,0), and (—1,1,—1). The linearization at the origin using 
variables u=2,v=y,w=zisu =u, vv = —v, 2’ =w. The linearization at the point 
(—1,1,-1) using variables u=a2+1,uv=y-l,w=z+lisw =u-2u, v = -v—2u, 
w’ =w — 2u. 


542s. eH flu) = ge) we = Th 
5.1.14: a) (0,0): saddle (unstable), (1,0): source (unstable), b) (0,0): spiral sink 


(asymptotically stable), (0,1): saddle (unstable), c) (1,0): saddle (unstable), (0,1): 
saddle (unstable) 


5.1.21: A critical point xo is stable if f’(ao) < 0 and unstable when f’(xo) > 0. 


5.2.2: a) $y? + 42° — 4x = C, critical points: (—2,0), an unstable saddle, and (2,0), a 
stable center. _b) sy? +e* =C, no critical points. c) sy? + xe* = C, critical point at 
(—1,0) is a stable center. 


5.2.3: Critical point at (0,0). Trajectories are y = +,/2C — (1/2)x4, for C > 0, these give 
closed curves around the origin, so the critical point is a stable center. 

5.3.2: a) Critical points are w = 0, 6 = km for any integer k. When k is odd, we have a 
saddle point. When k is even we get a sink. _b) The findings mean the pendulum will 
simply go to one of the sinks, for example (0,0) and it will not swing back and forth. The 
friction is too high for it to oscillate, just like an overdamped mass-spring system. 


bhtad a 
ac ’? 6b 


5.3.4: a) Solving for the critical points we get (0, —"/a) and ( 


). The Jacobian matrix 


at (0, —'/a) is aay, ° Hl whose eigenvalues are a + bh/d and —d. So the eigenvalues are 


always real of opposite signs and we get a saddle (In the application however we are only 


looking at the positive quadrant so this critical point is not relevant). At (ebad +) we get 


__ b(bh+ad) 


Jacobian matrix ne ppc | . b) For the specific numbers given, the second critical point is 
aro 


(3, 40) the matrix is | : ns ae which has eigenvalues SAV SRT Therefore there is a spiral 
source. This means the solution spirals outwards. The solution will eventually hit one of the 


axes, x = 0 or y = 0, so something will die out in the forest. 


5.3.5: The critical points are on the line x = 0. In the positive quadrant the 4’ is always 
positive and so the fox population always grows. The constant of motion is C = y%e~*~"Y, 
for any C this curve must hit the y-axis (why?), so the trajectory will simply approach a 
point on the y axis somewhere and the number of hares will go to zero. 


5.4.3: (0,0), unstable, r = /3, asymptotically stable. 
5.4.4: (0,0), asymptotically stable, r = V2, unstable, r = 2, asymptotically stable. 
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5.4.7: Use Bendixson—-Dulac Theorem. a) f + gy = 1+ 1 > 0, so no closed trajectories. b) 
fo + Jy = —sin*(y) + 0 < 0 for all x, y except the lines given by y = km (where we get zero), 
so no closed trajectories. c) fr + gy = y+0 > 0 for all x,y except the line given by y = 0 
(where we get zero), so no closed trajectories. 

5.4.8: Using Poincaré-Bendixson Theorem, the system has a limit cycle, which is the unit 
circle centered at the origin as x = cos(t) +e, y = sin(t) +e‘ gets closer and closer to the 
unit circle. Thus we also have that x = cos(t), y = sin(t) is the periodic solution. 

5.4.12: f(x,y) =y, 9(z,y) = w(l—2?)y—-z. So f,+9y = w(1—27). The Bendixson—Dulac 
Theorem says there is no closed trajectory lying entirely in the set x? < 1. 

5.4.14: The closed trajectories are those where sin(r) = 0, therefore, all the circles centered 
at the origin with radius that is a multiple of a are closed trajectories. 

5.5.1: Critical points: (0,0, 0), (38,38, 27), (—3V/8, —3\V8, 27). Linearization at (0, 0,0) 
using u=2,v=y,w=zisw = —10u+4 10v, v’ = 28u — v, w’ = —(8/3)w. Linearization 
at, (31/8, 3\/8, 27) using u = 2 — 3/8, v = y — 3V8, w = z — 27 is u’ = —10u+4 10v, 
v =u-—v—3vV8u, wv’ = 3V8u + 3/8u — (8/3)w. Linearization at (—3/8, —3V/8, 27) 
using u = 4 +378, v = y+ 3V8, w = z—- 27 is uw’ = —10u+ 10v, v' = u—v + 3V8u, 
w! = —3/8u — 3V/8u — (8/3) w. 
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